Entropy-based information fusion for multimodal data

Multimodal data contains a great amount of data in the Internet which hold rich-media content. The fusion of data information is a way to explore the linkage between the Web data in order to integrate the data from heterogeneous sources so that deep information can be extracted. Nowadays Web data are either structured or unstructured and information can be generated from the Web data by supervised or unsupervised methods. The existing methods rely on features generated from histogram data like HMM or pre-defined rules. However, data change in-deterministically at most of time and are hard to pre-define all the states and rules in advance. This paper takes the approach of entropy-based information fusion to treat the information from each source as a stochastic process so that the change of each process can be measured in real time. Then the change of information from each source is integrated and entropy is introduced to measure how far the integrated information of the change is from the best scenario based on histogram data. In such a way, it is possible to deduce an overall inference from data from difference sources in different presentations. Then the inference can be generated automatically with human interference.