Control of the trajectory of a hexapod robot based on distributed Q-learning

This paper presents a distributed approach of reinforcement learning used to learn a hexapod robot to control its trajectory according to a multilevel control decomposition. Locomotion functionality which consists in coordinating the legs so as to assure stable gait and in controlling the posture of the robot is more particularly investigated. As any leg cannot achieve its movements without interacting with others, coordination problems may occur. In order to take into account the actions of other agents, a distributed version of Q learning is proposed. The amplitudes of the movements are coded by self-organising maps and are adjusted during the training stage. The results of the simulation show that the robot can learn to control its trajectory efficiently.