Multi-modal information-sharing teaching system in sensor-based robotics $structured description based on task properties that enables onsite-tuning by reflecting skills

We consider a teaching system that covers the stages of task from planning to on-site teaching, and propose for it a structured description based on task properties. A multimodal communication-teaching-advisor (MCTA) system is used for on-site teaching for a tool-manipulating task. The modular structure and object-oriented task handling program enable it to be applied to various robot systems and lets system developers customize the interface easily. The interface allows the operator and robot to share internal and external sensor data via nonverbal media. The intentions of an inexpert operator, who has only common knowledge and ordinary abilities, can then be transferred to the robot system via nonverbal operation. Experiments show that two different types of robot controller could be controlled by this teaching system, and the nominal task program was designed for an environment with two controllers.