Cooperative multiagent reinforcement learning using factor graphs

In this paper, we propose a sparse reinforcement learning (RL) algorithm using factor graphs. The contribution is to make the original sparse RL algorithm applicable for tasks decomposed in a more general manner. For some problems, it is more reasonable to divide agents into cliques, each of which is responsible for its specific subtask. In this way, the global Q-value function is decomposed into the sum of simpler local Q-value functions, each of which may contain more than two action variables. Such decomposition can be expressed by a factor graph and exploited by the general max-plus algorithm to get the global greedy joint action. The experimental results show that our methodology is feasible and effective.