Hierarchical medical image annotation using SVM-based approaches

Automatic image annotation or image classification can be an important step when searching for images from a database. Common approaches to medical image annotation with the Image Retrieval for Medical Applications (IRMA) code make poor or no use of its hierarchical nature, where different dense sampled pixel based information methods outperform global image descriptors. In this work we address the problem of hierarchical medical image annotation by building a Content Based Image Retrieval (CBIR) system aiming to explore the combination of three different methods using Support Vector Machines (SVMs): first we concatenate global image descriptors with an interest points Bag-of-Words (BoW) to build a feature vector; second, we perform an initial annotation of the data using two known methods, disregarding the hierarchy of the IRMA code, and a third that takes the hierarchy into consideration by classifying consecutively its instances; finally, we make use of pairwise majority voting between methods by simply summing strings in order to produce a final annotation. Our results show that although almost all fusion methods result in an improvement over standalone classifications, none clearly outperforms each other. Nevertheless, these are quite competitive when compared with related works using an identical database.