Advanced telerobotic underwater manipulation using virtual fixtures and haptic rendering

As the variety and complexity of tasks carried about by telerobotic systems underwater increases, so does the demand for technology that increases efficiency and safety of operations. We present a novel approach to subsea manipulation that leverages the natural perceptive capability of human pilots. We present a method and framework for generation and rendering of six degree-of-freedom haptic virtual fixtures based on streaming point cloud data captured by an RGB-D sensor. Sensor data is captured at 30 Hz and virtual fixtures are constructed in real-time in parallel on a GPU. The method is applied to two typical underwater telerobotic valve operations by implementing a forbidden-region virtual fixture and a guidance virtual fixture. During operation, real-time haptic feedback is sent to the operator at 1000 Hz. This haptic feedback allows pilots of remotely-operated vehicles to receive sense of touch feedback through the use of non-contact sensors.