Isomorphic and sparse multimodal data representation based on correlation analysis

Multimodal data is more and more popular in recent years. It is most interesting and challenging to learn multimodal data representation which affects the performance of relevant applications greatly, such as retrieval and clustering. However, it is difficult to find an efficient representation for multimedia data of different modalities which are heterogeneous in low-level features. Also it is hard to bridge the semantic gap between features and semantics. In this paper, we propose an isomorphic and sparse multimodal data representation method. First, we learn an isomorphic content representation by analyzing kernel canonical correlation among heterogeneous features; secondly, we propose optimization strategy of graph-based semantic sparse boosting. Extensive experiments demonstrate the superiority of our method over several existing algorithms.