Semantically similar visual words discovery to facilitate visual invariance

A major limitation of many image classification and retrieval systems is that they only rely on the visual structure within images. However, images that have a different visual appearance may be semantically related at a higher level conceptualization. This paper presents a framework to deal with this problem by exploiting the well-known bag-of-visual words (BVW) model, to represent visual content. There are two key contributions of this paper. First, a novel approach for visual words construction is presented which takes the spatial information of keypoints into account in order to enhance the quality of visual words generated from extracted keypoints. Second, an approach to discover semantically similar visual word sets is proposed, which enables the BVW model to become invariant to certain changes in visual appearance. Consequently, the BVW model strengthens the discrimination power for visual content classification.