Switching Reinforcement Learning to mimic an infant´s motor development — Application to two-dimensional continuous action space

Reinforcement Learning (RL) attracts much attention as a technique of realizing computational intelligence such as adaptive and autonomous decentralized systems. In general, however, it is not easy to put RL into practical use. This difficulty includes a problem of designing a suitable action space of an agent, i.e., satisfying two requirements in trade-off: (i) to keep the characteristics (or structure) of an original search space as much as possible in order to seek strategies that lie close to the optimal, and (ii) to reduce the search space as much as possible in order to expedite the learning process. In order to design a suitable action space adaptively, we have proposed switching RL model to mimic a process of an infant\´s motor development in which gross motor skills develop before fine motor skills. Then, a method for switching controllers is constructed by introducing and referring to the "entropy". Further, through a computational experiment by using a robot navigation problem with two-dimensional continuous action space, the validity and the potential of the proposed method have been confirmed.