Combining multiple spatial hidden Markov models in image semantic classification and annotation

The spatial-hidden Markov model (SHMM) is a two dimensional generalization of the traditional hidden Markov model (HMM), with the capability of blockbased semantic annotation as well as classification of images. In this paper, we conduct a sensitivity analysis of SHMM in semantic classification with respect to different block sizes and from this analysis, we propose a novel multi-scales SHMM that combines multiple SHMMs, each classifying the image on a different scale. By regarding each SHMM as distinct classifiers, classifier combination algorithm can be applied to integrate the outputs of the respective SHMMs to improve image classification accuracy. Experiment results demonstrate that the multi-scale SHMM consistently outperforms single SHMMin image semantic classifications. The proposed approach can be extended to other block-based image classification algorithms.