An Adaptive Query Prototype Modeling Method for Image Search Reranking

As more and more images with user free tags are appearing on the Internet, image search reranking has received considerable attention to help users obtain images relevant to the query. In this paper, we propose to rerank the initial image search results using an adaptive online query modeling method based on local features. For a query and its initial rank list, we construct a visual word dictionary using the randomly selected images in the initial rank list. Then the top N images are incrementally used to evaluate the importance/score of the words for the query. The words with higher scores are selected to model the query. The relevance scores of the words to the query are also kept. Then the model is used to measure the relevance of each image in the image list to the query and rerank the image list according to the measured relevance. This method can obtain optimal local prototype for a query from the top N images (N is not large) without considering many images. Also, the relevances of the top N images to the query are not considered equally by weighting according to their positions in the initial image list. Using this model, the influences from some wrongly returned images at the top of the initial rank list can be filtered out. This method is also query independent and can be used to any other query. Experimental results on a large scale image dataset of Web Queries demonstrate the efficacy of the proposed method.