Multi-agent learning methods in an uncertain environment

In this paper, the multi-agent learning methods in an uncertain environment are addressed. The advantages and disadvantages of each algorithm are given. Rationality and convergence are the two main properties of multi-agent learning algorithms. However, it is very difficult to achieve both properties simultaneously. Minmax-Q learning is guaranteed to converge to equilibrium but there is no guarantee that this is the best response to the actual opponent. Therefore, Minmax-Q is not rational. In contrast, opponent modeling is rational but not convergent. Reinforcement learning using a variable learning rate and simultaneously achieves both properties. To reduce the dimension of state space, modular Q-learning and multilayered reinforcement learning are presented. The presented methods are not exhaustive, but they highlight the major methods used by researchers in the past years.