Connecting Language and Perception in Human-Robot Interaction

We present a new knowledge representation called a conceptual network model to create robots that engage In intuitive interaction with humans through natural language. The conceptual network connects natural language concepts with visual perceptions including color perception, shape perception, size perception, and spatial perception. In the implementation of spatial perception, we present a computational model based on spatial template theory to interpret qualitative spatial expressions. Based on the conceptual network model, our mobile robot can recognize the object that the user referred to through natural language. Experimental results show our approach promising.