Toward an assembly plan from observation. I. Task recognition with polyhedral objects

The authors present the assembly-plan-from-observation (APO) method for robot programming. The APO method aims to build a system that has the capability of observing a human performing an assembly task, understanding the task based on the observation, and subsequently generating a robot program to achieve the same task. This paper focuses on the task recognition module (TRM), the main component of a complete APO system. The TRM recognizes object configurations before and after an assembly task, detects a configuration transition, and infers the assembly task that causes such a configuration transition. We assume that each assembly task aims to achieve a face contact relation between an object that has just been manipulated and the stationary environmental objects. We prepare abstract task models that associate transitions of face contact relations with assembly tasks that achieve such transitions. Next, we implement TRM in order to verify two issues: 1) that such a contact transition can be recovered from the output of the object recognizer; and 2) that given these relation transitions, it is possible to use the abstract task models to generate robot motion commands