Development of an imitation behavior in humanoid Kenta with reinforcement learning algorithm based on the attention during imitation

Since an environment or body states of robots are very changeable, robotic imitation systems should have ability to develop their behaviors by themselves. For the development of the imitation behaviors, we assume that attention during imitation is the key information. In order to realize such imitation systems with evolving ability, the idea of reinforcement learning system based on the attention structure during imitation has been presented in this paper. First, for describing the attention during imitation behaviors, we define the term ´sensor-action attention pair´ as the pair of the most important sensor information and the focused body parts during that behavior. Second, we introduce R-learning, the reinforcement learning method for continual tasks such as imitation behaviors. Third, the method to design the state-action space and the reward function based on the sensor-action attention pair is proposed. At last, for the confirmation of the function of the proposed imitation behavior system, we have done some experiments using actual humanoid Kenta. In those experiments, Kenta can develop imitation behavior that imitates the hand position of the human.