A sensor-based dual-arm tele-robotic manipulation platform

This paper presents a novel telerobotic framework for human-directed dual-arm manipulation. Current telerobotic systems typically involve a single robot arm commanded by human through a joystick or a master arm. In contrast, our system involves a dual-arm robot manipulating a held object through human gestures without any mechanical coupling. Our experiment involves an industrial robot consisting of a torso, two seven degree-of-freedom arms, and two three-finger hands. We use the existing industrial robot controller, and only modify the position setpoint in the outer loop. The human interfaces to the robot using a set of gesture vocabulary. During object manipulation, the human gesture is interpreted as the desired configuration of the object. The robot performs autonomous vision-based target identification and alignment, grasp selection and force control, to ensure stable and robust object manipulation, with no demand on human for stable grasping. The heterogeneous components of the system are integrated through Robot Raconteur, a distributed communication and control software system. The system interfaces easily to powerful analysis and visualization tools, facilitating rapid algorithm development and prototyping. We envision that the integrated architecture will serve as the foundation for versatile, robust, and safe human-robot collaboration in increasingly complex sensing and manipulation tasks.