Visual query processing for efficient image retrieval using a SOM-based filter-refinement scheme

Visual query processing is one of the new issues for content-based image retrieval. In this paper, we propose (i) a filter-refinement scheme based on a modified form of the self-organizing map and (ii) a new interactive approach for similarity matching in image retrieval based on visual query processing. Specifically, we first propose a new local membership function, which preserves the relationships between the input feature vectors of the images and their neighboring weight vectors, to project the high dimensional input feature vectors to a low dimensional grid. Then, all the input feature vectors are mapped and visualized in the 2D grid. The feature vector of the query image is mapped and visualized in the 2D grid as well. The users not only can visualize the locations of the query image and the image data in the database, but also visualize the locations of the relevant and irrelevant images. Next, the users retrieve the candidates from the 2D grid interactively through visual query processing in the filter phase. Finally, the query results are obtained from the candidates by performing similarity ranking in the original feature space during the refinement phase. In order to accelerate the query process, we use a hierarchical tree to index the weight vectors of the self-organizing map (SOM) units to reduce the computation cost for finding the best matching unit. Our experiments show that (i) the proposed approach works well on both synthetic datasets and image data, (ii) the proposed visual query processing approach is more efficient than conventional approaches and can enhance the overall interactive experience through fast feedback, and (iii) the filter-refinement scheme makes our proposed approach more robust than conventional approaches.