A reinforcement learning model using macro-actions in multi-task grid-world problems

A macro-action is a typical series of useful actions that brings high expected rewards to an agent. Murata et al. have proposed an actor-critic model which can generate macro-actions automatically based on the information on state values and visiting frequency of states. However, their model has not assumed that generated macro-actions are utilized for leaning different tasks. In this paper, we extend the Murata´s model such that generated macro-actions can help an agent learn an optimal policy quickly in multi-task grid-world (MTGW) maze problems. The proposed model is applied to two MTGW problems, each of which consists of six different maze tasks. From the experimental results, it is concluded that the proposed model could speed up learning if macro-actions are generated in the so-called correlated regions.