Multimodal information joint learning for geotagged image search

With the explosion of social media on the Web, significant efforts have been dedicated to the research on social image retrieval and ranking. However, most existing social image ranking methods are disturbed by noisy tags which brings a strong need to find some complementary information for image ranking. Thanks to the rapid development of mobile devices, online social images increasingly attached with geographic locations. This presents new opportunities for social image ranking. In this paper, we propose a hypergraph-based framework which integrates image content, user-generated tags and geo-location information into image ranking problem. By representing each image as a vertex in the hypergraph, higher-order relationship among images can be reflected accurately. Furthermore, our framework simultaneously optimizes the ranking scores and hyperedge weights. Thus, the effects of different edges in the constructed hypergraph can be adaptively modulated. We evaluated our framework on a geotagged image dataset crawled from Flickr, the comparison results demonstrate the effectiveness of our method.