Title :
Integration of gaze and gesture detection in nature language instructing of robot in an assembly scenario
Author :
Zhang, Jianwei ; Baier, Tim ; Hueser, Markus
Author_Institution :
Dept. of Comput. Sci., Hamburg Univ., Germany
Abstract :
We present the development of and experiments with a robot system showing multimodal interaction capabilities. We focus on the understanding of human instructions in natural language by integrating gaze and pointing hand gestures. A typical application of such a system is interactive assembly. A human communicator sharing a view of the assembly scenario with the robot instructs the latter by speaking to it in the same way that he would communicate with a child. His instructions can be under-specified, incomplete and/or context-dependent. By detecting gaze and gesture information of the instructor, which are described in two sections of the paper, the ambiguities in the interactions can be resolved in a natural manner. Finally, we outline a list of future topics for extending our research.
Keywords :
Kalman filters; assembling; feature extraction; gesture recognition; interactive systems; natural language interfaces; robot programming; robot vision; 3D facial feature model; Kalman filter; assembly scenario; feature detection; gaze detection; gesture detection; human communicator; interaction ambiguities; interactive assembly; multimodal interaction capabilities; nature language instruction; pointing hand gestures; robot; stereo vision system; template matching process; two-arm robotic system; Aircraft; Assembly systems; Human robot interaction; Humanoid robots; Intelligent robots; Natural languages; Robot sensing systems; Robotic assembly; Robotics and automation; Speech recognition;
Conference_Titel :
Robot and Human Interactive Communication, 2002. Proceedings. 11th IEEE International Workshop on
Print_ISBN :
0-7803-7545-9
DOI :
10.1109/ROMAN.2002.1045629
