Combining haptic and visual servoing for cardiothoracic surgery

The primary goal of this research is to develop effective haptic and visual servoing methods, with the eventual goal of eliminating the need for mechanical stabilizers in a coronary artery bypass graft procedure by presenting a stationary operative site to the surgeon performing the procedure using haptic and visual feedback. We present the results from our initial work in the area of tracking a deformable membrane using vision and providing haptic feedback to the user based on the vision information and the material properties of the membrane. In our first experiment, we track the deformation of a rubber membrane in real-time through stereo vision while providing haptic feedback to the user interacting with the reconstructed membrane through the PHANToM haptic device. In the second experiment, we verify the ability of our vision system to track a point on a surface undergoing a complex 3D motion.