Grasp observation and reproduction by humanoid robots using color camera and 3D sensor

In this paper, we present a system for grasp observation, mapping and execution on humanoid robots to provide intuitive and natural way of communication between humans and robots. This system enables a human user to teach a robot how to grasp an object. The system includes four main components: the hand movement tracking component which provides the approach direction toward the object, the pre-grasp hand pose estimation component which provides the grasp type performed by the human user, the contact points estimation component which provides the position on the object should be grasped, and the grasp mapping and execution component for grasp reproduction on humanoid robots with gripper hands.