An improved Monte Carlo POMDPs online planning algorithm combined with RAVE heuristic

Partially observable Markov decision processes (POMDPs) provide a kind of general model that can deal with problems in uncertain environment efficiently. There are many different planning methods for POMDPs model. Partially Observable Monte Carlo planning (POMCP) method, using Monte Carlo Tree Search (MCTS) method, which can help break the curse of dimensionality and the curse of history. However, the method has strong dependence on the count of simulations. The POMCP algorithm was improved in this paper by combining Rapid Action Value Estimate (RAVE) method and MCTS. There´s less dependence on the count of simulations and higher efficiency in the improved algorithm, which is a promising online planning algorithm. Experimental results on the benchmark problems indicate that efficiency of the improved algorithm is higher than the basic POMCP algorithm.