Title :
The Power of Forgetting: Improving the Last-Good-Reply Policy in Monte Carlo Go
Author :
Baier, Hendrik ; Drake, Peter D.
Author_Institution :
Inst. of Cognitive Sci., Univ. of Osnabruck, Osnabrück, Germany
Abstract :
The dominant paradigm for programs playing the game of Go is Monte Carlo tree search. This algorithm builds a search tree by playing many simulated games (playouts). Each playout consists of a sequence of moves within the tree followed by many moves beyond the tree. Moves beyond the tree are generated by a biased random sampling policy. The recently published last-good-reply policy makes moves that, in previous playouts, have been successful replies to immediately preceding moves. This paper presents a modification of this policy that not only remembers moves that recently succeeded but also immediately forgets moves that recently failed. This modification provides a large improvement in playing strength. We also show that responding to the previous two moves is superior to responding to the previous one move. Surprisingly, remembering the win rate of every reply performs much worse than simply remembering the last good reply (and indeed worse than not storing good replies at all).
Keywords :
Monte Carlo methods; computer games; learning (artificial intelligence); tree searching; Monte Carlo Go; biased random sampling policy; forgetting; game playing; last-good-reply policy; simulated games; tree search; win rate; Algorithm design and analysis; Decision trees; Games; Machine learning; Monte Carlo methods; Board games; Go; Monte Carlo methods; machine learning;
Journal_Title :
Computational Intelligence and AI in Games, IEEE Transactions on
DOI :
10.1109/TCIAIG.2010.2100396