Region-based Q-learning for intelligent robot systems

It is desirable for autonomous robot systems to possess the ability to behave in a smooth and continuous fashion when interacting with an unknown environment. Since Q-learning requires a lot of memory and time to optimize a series of actions in a continuous state space, it may not be easy to apply the method to such a real environment. In this paper, for a continuous state space application, we propose a new method of Q-learning that incorporates a region-based reward assignment being used to solve a structural credit assignment problem and a triangular type Q-value model. Our learning method can estimate a current Q-value by a relationship with its neighboring states and has the ability to learn its actions similar to that of Q-learning. Thus, our method can enable robots to move smoothly in a real environment. To show the validity of our method, navigation comparison with Q-learning is given and visual tracking simulation results involving a 2-DOF SCARA robot are also presented