Technique for registering 3D virtual CT images to endoscopic video

We present a multi-modal image registration technique that matches virtual surface renderings, derived from 3D medical data, to corresponding endoscopic video. Our efforts focus on the chest. The views of the two image sources arise from inside the major bronchial airways, as shown in 3D CT chest images and live endoscopic video. Our technique is part of a large system for two-stage assessment of lung cancer. Stage-1 involves 3D CT assessment, which gives planning data for endoscopy. Stage-2 involves live endoscopy, supplemented by CT-based guidance. The registration technique is a critical component of this assessment. The technique draws upon the concept of normalized mutual information (NMI) and works in near real time on standard PCs. An optimization procedure iteratively adjusts the virtual viewpoint rendering until it matches a given video frame´s viewpoint. The optimization technique ends when the NMI is maximized. Results are given for phantom and human cases