Multimodal Medical Image Visualization Based on Mutual Information and Direct Multimodal Volume Rendering

Effective registration and rendering methods for multimodal 2D and 3D medical image visualization in various clinical applications are investigated. The proposed registration method was developed from the well-known fact in information theory that the mutual information (MI) is maximal when two images are geometrically aligned. The optimization of weighted average rendering (OWAR) and the direct multimodal volume rendering (DMVR) were proposed for 2D and 3D medical image visualization respectively. It was shown that MI criteria and DMVR suffered from the disadvantage of long time consumption although they can be applied automatically without any prior segmentation. For this reason, the multi-resolution optimization algorithm was used to decrease the computational complexity. Finally the proposed methods are examined by several numerical examples. The CT-MR and CT-PET image pairs were two-dimensionally visualized using the registration with MI criterion and the OWAR method. 256times256times225 CT and 256times256times167 MR volume data were three-dimensionally visualized using the DMVR method. The results demonstrate that OWAR method provides better display information and multi-resolution method reduces the processing time