The running control of humanoid robot utilizing Q-learning and output zeroing

This paper describes a control structure for humanoid runner which is using Q-learning and output zeroing. Based on this structure, the sagittal plane motion control algorithm is formulated using output zeroing. And Q-learning algorithm is used to learn key parameters of output function so that humanoid robot can achieve desired running motion. For the parameters of output function to be learned, the parameters which control the hip trajectory and torso angle of humanoid are selected. For an evaluation of learning, the running speed and stride of humanoid are used. As a result of learning, it is confirmed that key parameters of output function converge on constant value and that humanoid robot can realize continuous running in numerical simulation. Besides it is also verified that humanoid robot can realize continuous running at uneven terrain using similar control structure.