Dynamic fuzzy Q-learning control of uncertain systems with applications to humanoids

In this paper, a design methodology for enhancing the stability of humanoid robots is presented. Fuzzy Q-learning (FQL) is applied to improve the zero moment point (ZMP) performance by intelligent control of the trunk of a humanoid robot. With the fuzzy evaluation signal and the neural networks of FQL, biped robots are dynamically balanced in situations of time-varying terrains. Compared with scale reward, fuzzy evaluation gives more informative feedback for training. Different types of membership functions (MF) for fuzzy systems have been compared. Simulation studies show that the FQL controllers based on both triangular and Gaussian MFs are able to improve the stability as the actual ZMP trajectories become very close to the ideal case, and Gaussian function is superior as the actions generated are smoother and the learning speed is faster