Group utility functions: learning equilibria between groups of agents in computer games by modifying the reinforcement signal

Group utility functions are an expansion of the well known team utility function for providing multiple agents with a common reinforcement learning signal for learning collective behaviour. In this paper we describe what group utility functions are and use them with reinforcement learning to learn non-player character behaviours in a simple computer game. As yet, reinforcement learning techniques have rarely been used for computer game character behaviour specification. Using group utility functions, we can trade some optimality for some other desirable collective behaviour. As an example, in this paper we use group utility functions to learn an equilibrium between groups of agents performing a typical foraging task in a dynamic environment. Group utility functions act as filters on the reinforcement signal and sit between the reward function and the agents. We show several results demonstrating how group utility functions work in practice with varying learning parameters. An earlier paper describes our simpler initial results (Bradley et al., 2005). All our experiments are carried out using a commercial computer game engine.