Multi-robot Cooperation Based on Continuous Reinforcement Learning with Two State Space Representations

The field of multi-robot systems (MRS) deals with groups of autonomous robots and is attracting much research interest from robotics researchers. MRSs are expected to achieve tasks that are difficult to perform by an individual robot. In MRS, reinforcement learning (RL) is a promising approach for the distributed control of each robot. RL allows robots to learn mappings from their states to their actions through rewards or payoffs obtained through interactions with their environment. Theoretically, the environment of an MRS is nonstationary, and therefore rewards or payoffs received by learning robots depend not only on their own actions but also on the actions of other robots. From this perspective, our research group has been developing a technique that segments state and action spaces simultaneously and autonomously to extend the adaptability of an MRS to dynamic environments. To improve learning performance, we introduce a mechanism of selecting either of the two state spaces: one is represented by a parametric model for exploration, while the other is represented by a nonparametric model for exploitation. The proposed technique is expected to show better learning performance and robustness against an unpredicted environmental change. We investigate our proposed technique by conducting computer simulations of a cooperative box-pushing task with six autonomous mobile robots.