Reinforcement learning for motion control of humanoid robots

Many existing methods of reinforcement learning have treated tasks in a discrete low dimensional state space. However, the smooth control of humanoid robots requires a continuous high-dimensional state space. In this paper, to treat the state space, we proposed an adaptive allocation method of basis functions for reinforcement learning. Grid or incremental allocation methods have previously been proposed for allocation of basis functions. However, these methods may result in the curse of dimensionality, and a fall into local minima. On the other hand, our method avoids local minima, which are assessed by the trace of activity of basis functions. That is, if the current state is determined to have fallen into a local minimum, our method eliminates a basis function, which most affects the state. Moreover our method learns with a low number of basis functions because of the elimination process. In order to confirm the effectiveness of our method, by using computer simulation, a humanoid robot learned the motion of standing up from a chair. This motion was enabled with a small number of basis functions.