Surgical tools pose estimation for a multimodal HMI of a surgical robotic assistant

The main objective of this paper is to minimize the occluded areas in order to recognize the navigation of the surgeon´s tools for a two-arm autonomous robotic system for laparoscopic procedures. This robotic assistant needs the tracking of the surgeon´s surgical gestures in order to recognize the current maneuver and to execute the automated tasks of the robot. The surgical tools pose estimation is carried out by a Multiple Extended Kalman Filter (MEKF), where the movement models of the surgical tools depend on the maneuver which is being developed. This information is obtained by a maneuvers recognition system which is a part of the multimodal human machine interface (HMI) of the robot. The method proposed for reducing shadows has been applied to three in-vitro maneuvers which appear in the majority of the surgical protocols. The experiments show the behavior of this method for different time intervals of the occlusions.