An approach for mining heterogeneous data for cross-media retrieval

Due to the wide availability of huge amount of multimedia data in various modalities such as image and text documents, having a great amount of similarity among them is inevitable. In this paper, we present an efficient model which correlates the similarity among documents belonging to various modalities to achieve cross-media retrieval. Cross-media retrieval is a content based information retrieval system where heterogeneous data is mined to retrieve results of various modalities, i.e., input object and returned results may be of different modalities. For example, text objects can be retrieved as a result to image input. First, features are extracted from multimedia objects by which the objects are labeled. Using the labels, similar documents are grouped to generate Multimedia Documents. We construct a cross-media correlation graph with documents as vertices, where positive weight is assigned to every single edge according to the amount of similarity between vertices. The cross-media retrieval system identifies the input document and as a result returns required number of documents with highest weights.