Home appliance service system by using an object position and multimodal interaction with Communication robot

Gestures and utterances are useful factors in communication. A gesture or an utterance has different meanings depending on related objects, situation and so on. Authors aim that a system recognizes these interactions´ meanings and provides a service considering related objects. Proposed system combines gesture recognition and speech recognition as multimodal interaction, and manages the database with object position information through the interaction. Object position is one of the important factors for the system to recognize the meaning of human interaction. This paper focused on multimodal interaction between human and a robot to obtain object positions, and showed home appliance services as examples.