Flickr-based semantic context to refine automatic photo annotation

Automatic photo annotation task aims to describe the semantic content by detecting high level concepts in order to further facilitate concept based video retrieval. Most of existing approaches are based on independent semantic concept detectors without considering the contextual correlation between concepts. This drawback has its impact over the efficiency of such systems. Recently, harnessing contextual information to improve the effectiveness of concepts detection becomes a promising direction in such field. In this paper, we propose a new contextbased annotation refinement process. For this purpose, we define a new semantic measure called “Second Order Co-occurence Flickr context similarity” (SOCFCS) which aims to extract the semantic context correlation between two concepts by exploring Flickr resources (Flickr related-tags). Our measure is an extension of FCS measure by taking into consideration the FCS values of common Flickr related-tags of the two target concepts. Our proposed measure is applied to build a concept network which models the semantic context inter-relationships among concepts. A Random Walk with Restart process is performed over this network to refine the annotation results by exploring the contextual correlation among concepts. Experimental studies are conducted on ImageCLEF 2011 Collection containing 10000 images and 99 concepts. The results demonstrate the effectiveness of our proposed approach.