A Problem Case for UCT

Author

Browne, Cameron

Author_Institution

Dept. of Comput., Imperial Coll. London, London, UK

Volume

5

Issue

1

fYear

2013

fDate

Mar-13

Firstpage

69

Lastpage

74

Abstract

This paper examines a simple 5 × 5 Hex position that not only completely defeats flat Monte Carlo search, but also initially defeats plain upper confidence bounds for trees (UCT) search until an excessive number of iterations are performed. The inclusion of domain knowledge during playouts significantly improves UCT performance, but a slight negative effect is shown for the rapid action value estimate (RAVE) heuristic under some circumstances. This example was drawn from an actual game during standard play, and highlights the dangers of relying on flat Monte Carlo and unenhanced UCT search even for rough estimates. A brief comparison is made with RAVE failure in Go.

Keywords

Monte Carlo methods; computer games; estimation theory; search problems; trees (mathematics); RAVE failure; UCT performance; computer Go; domain knowledge; flat Monte Carlo search; rapid action value estimate heuristic; unenhanced UCT search; upper confidence bounds for tree search; Computer games; Estimation theory; Monte Carlo methods; Tree searching; Trees (mathematics); Bridge heuristic; Go; Hex; Monte Carlo methods; Monte Carlo tree search (MCTS); flat models; rapid action value estimate (RAVE) failure; upper confidence bounds (UCBs); upper confidence bounds for trees (UCT);

fLanguage

English

Journal_Title

Computational Intelligence and AI in Games, IEEE Transactions on

Publisher

ieee

ISSN

1943-068X

Type

jour

DOI

10.1109/TCIAIG.2012.2220138

Filename

6310042