From Robots to Reinforcement Learning

In this paper, we review recent advances in Reinforcement Learning (RL) in light of potential applications to robotics, introduce the basic concepts of RL and Markov Decision Process (MDP), and compare different RL algorithms such as Q-learning, Temporal Difference learning, the Actor Critic, and the Natural Actor Critic. We conclude that policy gradient methods are more suitable for solving continuous state/action MDP problems than RL with lookup tables or general function approximators. Further, natural policy gradient methods can efficiently converge to locally optimal solutions. Some simulation results are given to support our arguments. We also present a brief overview of our approach to developing an autonomous robot agent that can perceive, learn from and interact with the environment, and reason about and handle unexpected problems using its knowledge base.