Coordinating many agents in stochastic games

Learning in coordination games has been extensively studied in the game theory and multi-agent learning literature. Most of this work has considered a low number of agents and/or states (typically two agent, two action games). When the number of states and/or joint actions increases, standard approaches for multi-agent learning have difficulties coping with a high number of agents due to the combinatorial explosion in the number of joint actions and joint states. In real-world applications, this is a common setting though. This paper introduces a methodology for learning to coordinate in stochastic games with many agents. More specifically, we introduce a structure where some agents have knowledge about joint actions and how they have performed in the past. We empirically investigate this method for multi-agent learning in a typical stochastic game involving a high number of agents. Experimental results show that the additional information and structure is translated into earlier and higher levels of coordination and thus to higher payoffs.