Multimodal interaction for service robot control

In this paper we present the multimodal interaction of two sensor systems for control of a mobile robot. These systems consist of (1) an acoustic sensor that receives and recognizes spoken commands, and (2) a visual sensor that perceives and identifies commands based on the Mexican Sign Language (MSL). According to the stimuli, either visual or acoustic, the multimodal interface of the robotic system is able to weight each sensor´s contribution to perform a particular task. The multimodal interface was tested in a simulated environment to validate the pattern recognition algorithms (both, independently and integrated). The independent performance of the sensors was in average of 93.62% (visual signs and spoken commands), and of 95.60% for the multimodal system for service tasks.