Hidden Markov model-based multi-modal image fusion with efficient training

Automated spatial alignment of images from different modalities is an important problem, particularly in bio-medical image analysis. We propose a novel probabilistic framework, based on a variant of the 2D hidden Markov model (2D HMM), to capture the deformation between multi-modal images. Smoothness is ensured via transition probabilities of the 2D HMM and cross-modality similarity via class-conditional, modality-specific emission probabilities. The method is derived for general multi-modal settings, and its performance is demonstrated for an application in cellular microscopy. We also present an efficient algorithm for parameter estimation. Experiments on synthetic and real biological data show improvement over state-of-the-art multi-modal image fusion techniques.