Accelerating Q-Learning through Kalman Filter Estimations Applied in a RoboCup SSL Simulation

Speed of convergence in reinforcement learning methods represents an important problem, especially when the agent is interacting on adversarial environments like RoboCup Soccer domains. If the agent´s learning rate is too small, then the algorithm needs too many iterations in order to successfully learn the task, and this would probably lead to lose the game before the agent has learnt its optimal policy. We attempt to overcome this problem by using partial state estimations when some of the involved dynamics are known or easy to model for accelerating Q-learning convergence, illustrating the results in a RoboCup SSL simulation.