Markerless tracking for augmented reality for image-guided Endoscopic Retrograde Cholangiopancreatography

This paper proposes a markerless tracking method with adaptive pose estimation for augmenting 3D organ models on top of the endoscopic image for Endoscopic Retrograde Cholangiopancreatography (ERCP). While many applications of augmented reality (AR) to surgeries need special markers to track the camera´s position and orientation in the live video, our method employs the feature detection techniques to track the endoscopic camera. One of the most difficult problems when applying feature-based method to AR for ERCP is the lack of texture & highly specular reflection surface of duodenum in the endoscopic images, which does not provide a stable number of keypoints to track in the endoscopic video sequence. By introducing an adaptive weight function in the combination of reference-current frame tracking with previous-current frame tracking, we enhance the tracking performance remarkably. The proposed method is evaluated using an endoscopic video of a real ERCP and 3D duodenum model reconstructed from CT data of the patient. The result shows real-time performance and robustness of the method.