On Learning From Game Annotations

Most of the research in the area of evaluation function learning is focused on self-play. However in many domains, like Chess, expert feedback is amply available in the form of annotated games. This feedback usually comes in the form of qualitative information because human annotators find it hard to determine precise utility values for game states. The goal of this work is to investigate inasmuch it is possible to leverage this qualitative feedback for learning an evaluation function for the game. To this end, we show how the game annotations can be translated into preference statements over moves and game states, which in turn can be used for learning a utility function that respects these preference constraints. We evaluate the resulting function by creating multiple heuristics based upon different sized subsets of the training data and compare them in a tournament scenario. The results showed that learning from game annotations is possible, but, on the other hand, our learned functions did not quite reach the performance of the original, manually tuned function of the Chess program. The reason for this failure seems to lie in the fact that human annotators only annotate “interesting” positions, so that it is hard to learn basic information, such as material advantage from game annotations alone.