Robotic assembly operation teaching in a virtual environment

A “teaching by showing” system using a graphic interface is proposed. This system provides: 1) a user interface with which an inexperienced operator can easily teach a task, and 2) a task execution system that can operate the task in a different environment. The operator shows an example of assembly task movements in the virtual environmental created in the computer. A finite automaton dependently defined task is used to interpret the movements as a sequence of high-level representations of operations. When the system is commanded to operate the learning task, the system observes the task environment, checks the geometrical feasibility of the task in the environment, and if necessary replans the sequence of operations so that the robot can complete the task. Then the system uses task-dependent interpretation rules to translate the sequence of operations into manipulator-level commands and executes the task by replicating the operator´s movement in the virtual environment