Point Cloud-Based Model-Mediated Teleoperation With Dynamic and Perception-Based Model Updating

In this paper, we extend the concept of model-mediated teleoperation for complex environments and six degrees of freedom interaction using point cloud surface models. In our system, a time-of-flight camera is used to capture a high-resolution point cloud model of the object surface. The point cloud model and the physical properties of the object (stiffness and surface friction coefficient) are estimated at the slave side in real-time and transmitted to the master side using the modeling and updating algorithm proposed in this paper. The proposed algorithm adaptively controls the updating of the point cloud model and the object properties according to the slave movements and by exploiting known limitations of human haptic perception. As a result, perceptually irrelevant transmissions are avoided, and thus the packet rate in the communication channel is substantially reduced. In addition, a simple point cloud-based haptic rendering algorithm is adopted to generate the force feedback signals directly from the point cloud model without first converting it into a 3-D mesh. In the experimental evaluation, the system stability and transparency are verified in the presence of a round-trip communication delay of up to 1000 ms. Furthermore, by exploiting the limits of human haptic perception, the presented system allows for a significant haptic data reduction of about 90% for teleoperation systems with time delay.