Multi-robot collaboration based on Markov decision process in Robocup3D soccer simulation game

Close collaboration and desired strategy is indispensable for humanoid robots in the RoboCup soccer competition. In order to solve the problem that the convergence rate is too low in training local strategies, this paper mainly proposed a method to optimize the parameters in decision and positioning based on reinforcement learning for soccer robots. First, Markov decision process is applied to the framework for reinforcement learning. Then, we propose a relative improved method, which is known as a Sarsa Algorithm to overcome the drawback of the low convergence rate of the average reward reinforcement learning. Meanwhile, in order to deal with the large state space problems arising in the training and improve the generalization ability, this method is applied to the Keepaway local training. The training results show that, this algorithm has a faster convergent speed than other ordinary learning algorithm.