A Q-learning based Cartesian model reference compliance controller implementation for a humanoid robot arm

This paper presents the implementation (real time and simulation) of a model-free Q-learning based discrete model reference compliance controller for a humanoid robot arm. The Reinforcement learning (RL) scheme uses a recently developed Q-learning scheme to develop an optimal policy on-line. The RL Cartesian (x and y) tracking controller with model reference compliance was implemented using two links (shoulder flexion and elbow flexion joints) of the right arm of the humanoid Bristol-Elumotion-Robotic-Torso II (BERT II) torso.