Mean-variance based ranking and selection

The traditional approach in ranking and selection procedures is to compare simulated systems based on the mean performance of a metric of interest. The system with the largest (or smallest) mean performance is deemed as the best system. However, the system with the best mean performance may be an undesirable choice because of its large variance. Variance is a measure of risk. A highly variable system performance shows that the system is not under control. Both mean and variance of a performance metric need to be compared to determine the best system. We present a statistically valid selection procedure for comparing simulated systems based on a mean-variance dominance relationship. The system with the best mean and smallest variance is deemed as the best system. If there is not a unique best system, the procedure identifies a set of nondominant systems. In both cases, a prespecified probability of correct selection is guaranteed.