Image tag refinement along the ‘what’ dimension using tag categorization and neighbor voting

Online sharing of images is increasingly becoming popular, resulting in the availability of vast collections of user-contributed images that have been annotated with user-supplied tags. However, user-supplied tags are often not related to the actual image content, affecting the performance of multimedia applications that rely on tag-based retrieval of user-contributed images. This paper proposes a modular approach towards tag refinement, taking into account the nature of tags. First, tags are automatically categorized in five categories using WordNet: `where´, `when´, `who´, `what´, and `how´. Next, as a start towards a full implementation of our modular tag refinement approach, we use neighbor voting to learn the relevance of tags along the `what´ dimension. Our experimental results show that the proposed tag refinement technique is able to successfully differentiate correct tags from noisy tags along the `what´ dimension. In addition, we demonstrate that the proposed tag refinement technique is able to improve the effectiveness of image tag recommendation for non-tagged images.