A Study of Quality Issues for Image Auto-Annotation With the Corel Dataset

The Corel Image set is widely used for image annotation performance evaluation although it has been claimed that Corel images are relatively easy to annotate. The aim of this paper is to demonstrate some of the disadvantages of datasets like the Corel set for effective auto-annotation evaluation. We first compare the performance of several annotation algorithms using the Corel set and find that simple near neighbor propagation techniques perform fairly well. A support vector machine (SVM)-based annotation method achieves even better results, almost as good as the best found in the literature. We then build a new image collection using the Yahoo Image Search engine and query-by-single-word searches to create a more challenging annotated set automatically. Then, using three very different image annotation methods, we demonstrate some of the problems of annotation using the Corel set compared with the Yahoo-based training set. In both cases the training sets are used to create a set of annotations for the Corel test set