Concept Constrained Image Region Annotation

Annotating image regions has been a challenging open issue in many areas such as image content understanding and image retrieval. In this paper, rather than solely rely on visual features of image regions, a novel approach is proposed to improve region annotation by taking concept constraints into account, since high level conceptual information such as image categories can increase the confidence of possible region labels as well as decrease the confidence of impossible region labels. We employ statistical models to learn the relationships among visual features, image concepts, and region labels. As a result, a set of possible region labels can be derived from a set of visual feature vectors of a given image so as to refine the annotation output obtained by using visual feature only. Promising experimental results have been demonstrated on 8462 regions of the University of Washington image dataset with diverse concepts for the proposed approach.