Query-Adaptive Image Search Re-ranking Using Deep Convolutional Neural Network Feature

Image search re-ranking, as an effective tool to improve the text-based image search result, has been adopted by many commercial search engines nowadays. Given a query keyword, images are first retrieved based on the textual information. Then visual features are extracted from images to reorder them by mining their visual patterns. However, the popular visual features applied in re-ranking are not informative enough. Besides, the parameters for the re-ranking models are set equally for all queries, which fails to cope with the variability of different queries. In this paper, we propose a novel re-ranking method which adopts informative visual features for image representation and adaptively re-rank the images. Specifically, we adopt a proven successful DCNN feature (deep convolutional neural network), which shows the excellent performance in many computer vision fields, to calculate the visual similarities between images. For each query, the parameters for the image search re-ranking model is adaptively determined using the QDE (query difficulty estimation) method. Experiments are conducted on the INRIA web353 dataset. The experimental results demonstrate that our method achieves significant improvement over state-of-the-art methods.