Two hands are better than one: Assisting users with multi-robot manipulation tasks

Multi-robot manipulation, where two or more robots cooperatively grasp and move objects, is extremely challenging due to the necessity of tightly coupled temporal coordination between the robots. Unfortunately, introducing a human operator does not necessarily ameliorate performance due to the complexity of teleoperating mobile robots with high degrees of freedom. The human operator´s attention is divided not only among multiple robots but also between controlling a robot arm and its mobile base. This complexity substantially increases the potential neglect time, since the operator´s inability to effectively attend to each robot during a critical phase of the task leads to a significant degradation in task performance. In this paper, we propose an approach for semi-autonomously performing multi-robot manipulation tasks and demonstrate how our user interface reduces both task completion time and the number of dropped items over a fully teleoperated robotic system. Propagating the user´s commands from the actively-controlled robot to the neglected robot allows the neglected robot to leverage this control information and position itself effectively without direct human supervision.