Evolving Stochastic Controller Networks for Intelligent Game Agents

It is sometimes useful to provide intelligent agents with some degree of stochastic behavior, particularly when used in games and simulators. The less-predictable behavior that results from the randomness can make the agents seem more believable, and would encourage the players or users to address the genuine problems presented by a game or simulator rather than simply learning to exploit the embedded agents´ predictability. However, such randomized behavior should not harm performance in the agents´ designated tasks. This paper introduces a method, called stochastic sharpening, for training artificial neural networks as stochastic controllers for agents in discrete-state environments. Stochastic sharpening reinforces the representation of confidence values in the outputs of networks with localist encodings, and thus produces networks that recommend alternative actions on the basis of their expected utility. Such networks can be used to introduce stochastic behavior with minimal disruption of task performance, resulting in agents that are more believable and less subject to exploitation based on predictability.