Role differentiation process by division of reward function in multi-agent reinforcement learning

There are several problems which discourage an organization from achieving tasks, e.g. partial observation, credit assignment, and concurrent learning in the multi-agent reinforcement learning domain. In many conventional approaches, each agent estimates hidden states, e.g., other agentspsila sensor inputs, positions, and policies, and reduces the uncertainty in the partially-observable Markov decision process (POMDP), and solves the multiagent reinforcement learning problem. In contrast, people reduce uncertainty in human organizations in the real world by autonommously dividing the roles played by each agent. In a framework of reinforcement learning, roles are mainly represented by goals for each agent. This paper presents a method for generating internal rewards from manager agents to worker agents. It also explicitly divides the roles, which can change a POMDP task for each agent into a simple MDP task under certain conditions. Several situational experiments are also described and the validity of the proposed method is evaluated.