Chaotic switching in a two-person game

We study robust long-term complex behaviour in the Rock–Scissors–Paper game with two players, played using reinforcement learning. The complex behaviour is connected to the existence of a heteroclinic network for the dynamics. This network is made of three heteroclinic cycles consisting of nine equilibria and the trajectories connecting them. We provide analytical proof both for the existence of chaotic switching near the heteroclinic network and for the relative asymptotic stability of at least one cycle in the network, leading to some trajectories paying alternate visits to only two nodes while others follow a complicated sequence of visits to all nodes in the network. Our results are obtained by making use of the symmetry of the original problem, a new approach in the context of learning, and they provide an explanation of numerical results previously obtained by other authors.