Robust forceps tracking using online calibration of hand-eye coordination for microsurgical robotic system

Advanced robotic assistance in microsurgery, such as automation, requires an accurate estimation of the state of the robotic forceps. In this paper, we propose a robust and accurate forceps tracking method to estimate the full state of the forceps (i.e., the position, posture, and grip parameters) using visual information obtained from stereo microscopic images and kinematic information obtained from the robotic sensory information, forward kinematics, and hand-eye coordination. An online method for updating the hand-eye coordination was also developed using an extended Kalman filter to cancel the hand-eye coordination errors caused by the repositioning of the microscope. The experimental results showed that the proposed method could accurately and robustly estimate the state of the robotic forceps even after the repositioning of the microscope.