Option and constraint generation using Work Domain Analysis

In this paper we investigate the use of Work Domain Analysis (WDA), a technique from the field of cognitive engineering, to inform the creation of options and constraints for Reinforcement Learning (RL) algorithms. The micro-world of Pac-Man, a classic arcade game, is used as a tractable and representative work domain. WDA was conducted on individuals familiar with Pac-Man and an Abstraction Hierarchy (AH), a means-ends representation of their understanding of the game, was created for each individual. The abstraction hierarchies for best performing and worst performing individuals were then combined to illustrate the differences between the different groups. Several differences between the two groups were found, and included the use of defense as well as offensive strategies by high performers versus only defense by poor performers, context sensitivity and additional goals and more sophisticated constraints by high performers. The differences were translated into an options and constraint paradigm suitable for incorporation into RL algorithms.