Efficient computing budget allocation for simulation-based policy improvement

The dynamics of many systems in nowadays follow not only physical laws but also man-made rules. These systems are known as discrete event dynamic systems (DEDS´s). Policy improvement of such systems is usually based on simulation, which is time consuming and provides only noisy performance evaluation. It is of great practical interest to understand how to allocate the computing budget among action candidates so that a better policy is found with high probability. Despite the abundant studies on simulation-based policy optimization, few consider this allocation problem. This paper considers this important problem. Based on the method of optimal computing budget allocation (OCBA) in simulation-based optimization, an efficient allocation procedure is developed, which is shown to asymptotically maximize a lower bound of the probability of correctly selecting the best action. This allocation procedure is compared with equal allocation, which is well adopted in practice, on numerical examples. The numerical results show that even when there are only finite computing budget to allocate, this OCBA-based allocation procedure outperforms equal allocation and has good performance. We hope this work brings insight to computing budget allocation for simulation-based policy improvement in more general situations.