Gait Synthesis Self-generation by Dynamic Fuzzy Q-Learning Control of Humanoid Robots

This paper introduces a novel self-generated gait synthesis approach for dynamic control of Humanoid Robots (HRs). The main idea is to define gait trajectories for hip and ankle so that motions of other joints can be regulated simultaneously. Because stability is one of the most common concerns requested for HRs, a self-learning control strategy of improving dynamic stability based on the zero moment point (ZMP) criterion is developed. As hip motion plays the most important role of dynamic stability, a dynamic fuzzy Q-learning (DFQL) controller is proposed to define the hip motion trajectory. A salient feature of the proposed approach is that the DFQL controller can self generate fuzzy rules without a priori knowledge and it is capable of dealing with dynamic systems. The DFQL controller can automatically generate the structure as well as parameters of the fuzzy system. Simulation results show that the DFQL controller is capable of improving dynamic stability as the actual ZMP trajectory becomes very close to the ideal case.