Forbidden-region virtual fixtures from streaming point clouds: Remotely touching and protecting a beating heart

Several established methods for remote touching using non-contact sensors exist. Applications for these methods are primarily within the field of robotic teleoperation. In surgical robotics it would be useful to not only touch, but also be able to maintain a distance from a certain organ. The latter can be done using non-contact sensors such as cameras. The novelty in this paper is the idea of combining forbidden-region virtual fixtures with haptic rendering from streaming point clouds. This is then used to protect as well as remotely touch a beating heart without any a priori knowledge of the heart geometry (such as from CT/MR scans).