Patch based specular reflection removal for range images in hybrid 3-D endoscopy

In minimally invasive surgery, hybrid 3-D endoscopy is an evolving field of research that aims to augment conventional video based systems by metric 3-D measurements. One crucial issue with these systems arises from specular reflections that result in overexposed RGB values and invalid range measurements. In this paper, we address this problem by registering video and range information acquired from different view points using a patch based approach. This allows to replace invalid measurements caused by specular reflections in one view with valid data from non-specular regions in the other view. In contrast to previous approaches that employ interpolation techniques our method utilizes actual scene information being advantageous in a medical environment. In our experiments, we show that our method decreases the mean absolute error in common situations on average by more than 30% compared to conventional interpolation. Moreover, for challenging scenarios we outperform interpolation by more than 1 mm and reconstruct important structures that inherently could not be restored by conventional interpolation.