A Multi-Modal Topic Model for Image Annotation Using Text Analysis

Most of the existing approaches for image annotation generally demand exactly labeled training data, which are often difficult to obtain. In this letter we present a novel model that utilizes the rich surrounding text of images to perform image annotation. Our work makes two main contributions. First, by integrating text analysis, words that describe the salient objects in images are extracted. Second, a new probabilistic topic model is built to jointly model image features, extracted words and surrounding text. Our model is demonstrated to be flexible enough to handle multi-modal features and provide better performance than the state-of-the-art annotation methods.