Alphabet-Based Multisensory Data Fusion and Classification Using Factor Graphs

The way of multisensory data integration is a crucial step of any data fusion method. Different physical types of sensors (optic, thermal, acoustic, or radar) with different resolutions, and different types of GIS digital data (elevation, vector map) require a proper method for data integration. Incommensurability of the data may not allow to use conventional statistical methods for fusion and processing of the data. A correct and established way of multisensory data integration is required to deal with such incommensurable data as the employment of an inappropriate methodology may lead to errors in the fusion process. To perform a proper multisensory data fusion several strategies were developed (Bayesian, linear (log linear) opinion pool, neural networks, fuzzy logic approaches). Employment of these approaches is motivated by weighted consensus theory, which lead to fusion processes that are correctly performed for the variety of data properties. As an alternative to several methods, factor graphs are proposed as a new approach for multisensory data fusion. Feature extraction (data fission) is performed separately on different sources of data to make an exhausting description of the fused multisensory data. Extracted features are represented on a finite predefined domain (alphabet). Factor graph is employed for the represented multisensory data fusion. Factorization properties of factor graphs allow to obtain an improvement in accuracy of multisensory data fusion and classification (identification of specific classes) for multispectral high resolution WorldView-2, TerraSAR-X SpotLight, and elevation model data. Application and numerical assessment of the proposed method demonstrates an improved accuracy comparing it to well known data and image fusion methods.