Concurrently learning neural nets: encouraging optimal behavior in cooperative reinforcement learning systems

Reinforcement learning agents interacting in a common environment often fail to converge to optimal system behaviors even when the individual goals of the agents are fully compatible. Claus and Boutilier have demonstrated that the use of joint action learning helps to overcome these difficulties for Q-learning systems. This paper studies an application of joint action learning to systems of neural networks. Neural networks are a desirable candidate for such augmentations for two reasons: (1) they may be able to generalize more effectively than Q-learners, and (2) the network topology used may improve the scalability of joint action learning to systems with large numbers of agents. Preliminary results indicate that neural nets benefit from joint action learning in the same way that Q-learners do.