Multimodal Data Fusion Using Source Separation: Two Effective Models Based on ICA and IVA and Their Properties

Fusion of information from multiple sets of data in order to extract a set of features that are most useful and relevant for the given task is inherent to many problems we deal with today. Since, usually, very little is known about the actual interaction among the data sets, it is highly desirable to minimize the underlying assumptions. This has been the main reason for the growing importance of data-driven methods, and in particular of independent component analysis (ICA) as it provides useful decompositions with a simple generative model and using only the assumption of statistical independence. A recent extension of ICA, independent vector analysis (IVA), generalizes ICA to multiple data sets by exploiting the statistical dependence across the data sets, and hence, as we discuss in this paper, provides an attractive solution to fusion of data from multiple data sets along with ICA. In this paper, we focus on two multivariate solutions for multimodal data fusion that let multiple modalities fully interact for the estimation of underlying features that jointly report on all modalities. One solution is the joint ICA model that has found wide application in medical imaging, and the second one is the transposed IVA model introduced here as a generalization of an approach based on multiset canonical correlation analysis. In the discussion, we emphasize the role of diversity in the decompositions achieved by these two models, and present their properties and implementation details to enable the user make informed decisions on the selection of a model along with its associated parameters. Discussions are supported by simulation results to help highlight the main issues in the implementation of these methods.