Best-response search algorithms for non-stationary discrete stochastic optimization

This paper considers simulation-based optimization of the performance of a regime switching stochastic system over a finite set of feasible alternatives. We propose an adaptive simulation-based search algorithm that tracks the randomly switching set of global optima, and has two key features: (i) it allows temporal correlation in simulation data; (ii) it distributes the search such that most of the simulation experiments are performed on the set of global optimizers. Numerical examples verify that the proposed scheme yields faster convergence for finite sample lengths compared with existing random search and pure exploration methods.