Implications of heavy tails on simulation-based ordinal optimization

We consider the problem of selecting the best system using simulation-based ordinal optimization. This problem has been studied mostly in the context of light-tailed distributions, where both Gaussian-based heuristics and asymptotically optimal procedures have been proposed. The latter rely on detailed knowledge of the underlying distributions and give rise to an exponential decay of the probability of selecting the incorrect system. However, their implementation tends to be computationally intensive. In contrast, in the presence of heavy tails the probability of selecting the incorrect system only decays polynomially, but this is achieved using simple allocation schemes that rely on little information of the underlying distributions. These observations are illustrated via several numerical experiments and are seen to be consistent with asymptotic theory.