Robot vision to recognize both object and rotation for robot pick-and-place operation

This paper presents a robot vision system to recognize both different target objects and their poses that can be incorporated in robot programming by demonstration (robot PbD). In recent years, robot PbD has become a popular topic in the field of robotics. Programming robots is a time-consuming process and requires technical knowledge. However, robot task programming will be simplified by using a vision system to analyze human demonstration. Usually, design of a user-friendly human-robot interaction interface is crucial to implement the robot PbD process. We take pick-and-place operation as an example of robot PbD. Several objects are placed on a table without overlapping, and people pick one of them to other place, one after another. Before learning from human demonstration, if the robot can obtain initial information, e.g., to recognize the target object and to estimate its pose, it helps robot to refine its following decision, such as selecting the gripper type or the grip angle. Thus the authors develop the robot vision ads the first step to realize such working robot. The developed vision system has advantage of interactive trainability with the help of the proposed graphical user interface (GUI), in which the interpretable features can be easily selected by ordinary users. We also propose a simple scale-invariant pose estimation method. Experimental results support feasibility and effectiveness of the system.