Self-segmentation of sequences

The paper presents an approach for hierarchical reinforcement learning that does not rely on a priori hierarchical structures. Thus the approach deals with a more difficult problem compared with existing work. It involves learning to segment sequences to create hierarchical structures, based on reinforcement received during task execution, with different levels of control communicating with each other through sharing reinforcement estimates obtained by each others. The algorithm segments sequences to reduce non-Markovian temporal dependencies, to facilitate the learning of the overall task. Initial experiments demonstrated the basic promise of the approach