Task world reality for human and robot system. A multimodal teaching advisor and its implementation

The concept of task world reality (TWR) for a sensor-based robot system and human operator at the work-site is proposed. Based on the TWR concept, a multimodal teaching advisor (MTA) which utilizes the work-site human operator´s know-how and sensory information in a complementary manner has been developed. This system integrates task specifications, system constraints including their tolerances, sensing information, and operator- and robot-related information. MTA provides both graphical and audio user interfaces using a spatial human trader in order to support an advanced teaching environment for a work-site operator. A complicated spatial path tracking task is taken as a typical case study. Experimental results show that not only the human operator´s teaching load but also the task quality are greatly improved by MTA