Creating intelligent agents through shaping of coevolution

Creating agents that behave in complex and believable ways in video games and virtual environments is a difficult task. One solution, shaping, has worked well in evolution of neural networks for agent control in relatively straightforward environments such as the NERO video game, but is very labor intensive. Another solution, coevolution, promises to establish shaping automatically, but it is difficult to control. Although these two approaches have been used separately in the past, they are compatible in principle. This paper shows how shaping can be applied to coevolution to guide it towards more effective behaviors, thus enhancing the power of coevolution in competitive environments. Several automated shaping methods, based on manipulating the fitness function and the game rules, are introduced and tested in a "capture-the-flag"-like environment, where the controller networks for two populations of agents are evolved using the rtNEAT neuroevolution method. Each of these shaping methods as well as their combinations are superior to a control, i.e. direct evolution without shaping. They are effective in different and sometimes incompatible ways, suggesting that different methods may work best in different environments. Using shaping, it should thus be possible to employ coevolution to create intelligent agents for a variety of games.