Integrating whole body motion primitives and natural language for humanoid robots

This paper describes the novel approach to integration of natural language processing and symbolization of motion patterns in order to allow for humanoid robotpsilas acquisition of language. This framework consists of two models : motion language model and natural language model. In the motion language model, morpheme words are stochastically associated with symbolized motion patterns via latent variables. The association is defined by probability that the motion pattern generates the latent variable and probability that the latent variable generates the morpheme word. The natural language model represents order relation among the morpheme words via word classes by using hidden Markov model. The motion language model and the natural language model are equivalent to semantics and syntax respectively. Integration of the motion language model and the natural language model achieves linguistic interpretation of motion patterns by composing semantically and syntactically appropriate sentence. The efficient algorithm for the composition is proposed. The validity of the motion language model, the natural language model and the integration is demonstrated by testing the implemented algorithm on human motion data.