Task-oriented teleoperation through natural 3D user interaction

Teleoperation provides a technical means to perform desired tasks in remote environments. Teleoperation using direct control and haptic-mediated interactions requires significant effort for unskilled users to understand how to operate the robot. Task-oriented teleoperation supports human-level understanding to directly transfer the meaning of tasks from the user to the robot. In this paper, we propose a natural 3D interface to transfer task knowledge to a remote robot. We design a 3D manipulation system in a mixed reality environment that combines human hand gestures and a 3D view of the remote robot. We demonstrate that the remote robot successfully executes the ordered task even in dynamic environments.