Vision-speech system adapting to the user and environment for service robots

This paper presents a vision-speech system for service robots that can learn the user´s customs and objects fixed in the environment while helping the user, and can perform their tasks more efficiently with less user´s burden. We are working on a service robot that fetches the objects ordered by the user through speech. The robots need vision to recognize the objects. It asks the user for help by speech if its vision fails. In early stages, it asks the user for help many times and requires time to detect the objects by vision, because it does not know much about the environment. We assume that humans usually put a particular object on a small number of places. The robot obtains such information and the environmental knowledge while accomplishing the user´s demands. In later stages, the user does not need to say details because the robot knows where the objects usually are. Moreover, the vision processing time is greatly reduced because it knows what operations can work in such cases. Experiments using a robot system show the usefulness of the proposed system.