Image tag refinement using tag semantic and visual similarity

Social tagging on online websites provides users interfaces of describing resources with their own tags, and vast user-provided image tags facilitate image retrieval and management. However, these tags are often not related to the actual image content, affecting the performance of tag related applications. In this paper, a novel approach to automatically refine the image tags is proposed. Firstly, information entropy of the tag is defined to refine tag frequency to predict tag initial relevance. Then, tag correlation is calculated from two sides. One side is to measure semantic similarity of tag pairs using the structured information of the free encyclopedia Wikipedia. The other one is to compute the visual similarity of tag pairs based on the visual representation of the tag. Finally, to re-rank the original tags, a fast random walk with restart is used and the top ones are reserved as the final tags. Experimental results conducted on dataset NUS-WIDE demonstrate the promising effectiveness of our approach.