Reducing Exploration by Closing Enclosures in Cluttered State Space

In cluttered state space, agent is very likely to become trapped in these enclosures formed by obstacles. We proposed an approach to discover the openings of enclosures based the state trajectories that agent moved, then the openings of enclosures are closed by virtual obstacles to avoid agent entering these enclosures again in subsequent learning. Since large amount of episodes are included in learning process, the accumulative reduced exploration by our method are significant. Moreover, agent would have higher learning efficiency due to avoid cycle behavior in these enclosures. The experiments on Maze problem show the applicability and effectiveness of our method.