Estimation of other´s sensory patterns based on dialogue and shared motion experiences

Estimating another´s or system´s internal sensory patterns such as torque, sound and vision would be an important capability for advanced intelligent human-robot interaction systems. In particular, somato-sensory patterns are hard to estimate because specific musculo-skeletal models and electromyography (EMG) cannot be used in natural environments. In this paper, we propose an approach by which a humanoid robot imitates other´s motion to estimate the other´s sensory pattern by using a mirror system and dialogue. The robot uses the mirror system to associate the other´s unobservable internal sensory patterns with its own sensory patterns. If the body conditions of the self are different from those of the other, the estimated sensory pattern would be different from the true pattern. In such case, the robot can try to cancel out the differences by modifying the sensor-motion association map of the other, so that the observable motions can be transformed into an unobservable sensory pattern. By engagin in dialogue, the robot can acquire the sensor-motion association map, even if the sensor pattern is unobservable. We developed such a sensormotion association map model and a symbol communication function to estimate others´ sensor patterns from dialogues based on shared motions. Experiments on virtual humanoid robots confirmed the feasibility of this symbol-communication-based estimation method.