Effective multimodal information fusion by structure learning

The joint processing of multimodal data received a lot of attention in the last decade due to the increased availability of multimedia data and the under-performance of content-based approaches. This work shows that effective information fusion for multimedia document processing can be achieved by learning the data structure that underlies the fusion task and adapting the fusion strategy accordingly. The approach of structure learning combined with fusion strategy adaption is implemented in the feature selection and construction (FS/FC) algorithm. The effectiveness of the approach is shown on behalf of a boolean concept and a multimedia document classification task.