Shrinking large visual vocabularies using multi-label agglomerative information bottleneck

The quality of visual vocabularies is crucial for the performance of bag-of-words image classification methods. Several approaches have been developed for codebook construction, the most popular method is to cluster a set of image features (e.g. SIFT) by k-means. In this paper, we propose a two-step procedure which incorporates label information into the clustering process by efficiently generating a large and informative vocabulary using class-wise k-means and reducing its size by agglomerative information bottleneck (AIB). We introduce an extension of the AIB procedure for multi-label problems and show that this two-step approach improves the classification results while reducing computation time compared to the vanilla k-means. We analyse the reasons for the performance gain on the PASCAL VOC 2007 data set.