Automatic option generation in hierarchical reinforcement learning via immune clustering

An open problem in hierarchical reinforcement learning is how to automatically generate hierarchies, e.g. options. We consider an immune clustering approach for automatic construction of options in a dynamic environment. The learning agent generates an undirected edge-weighted topological graph of the environment state transitions online. An immune clustering algorithm is then used to partition the state space. A second immune response algorithm is used to update the clusters when a new state being encountered later. Local strategies for reaching the different parts of the space are separately learned and added to the model in a form of options. By our approach, the options not only can be automatically generated but also can be dynamically updated