A new approach to credit assignment in a team of cooperative Q-learning agents

Q-learning is widely used in many multi agent systems. In most cases, a separate critic is considered for qualifying each individual agent behavior or it is assumed that the critic is completely aware of effects of all agents´ actions on the team qualification. But, in many cases, the role of each team member in the group performance is not known. In order to distribute a common credit among the agents, a suitable criterion must be provided to estimate the role of each agent in the team performance and to judge if an agent has done a wrong action. In this paper two such criteria, named certainty and expertness, for a team of agents with parallel tasks are introduced. In addition, two methods for reinforcing the agents based on the proposed measures are provided. Some simulation results are also reported to show the effectiveness of the proposed measures and methods.