Parameter optimization by means of statistical quality guides in F-Race

F-Race and its variant, Iterated F-Race, is an automated procedure for sampling and evaluating potential values of parameters for algorithms. The procedure is controlled by means of a computational budget that limits the number of evaluations that may be conducted, thus forcing the determination of the best possible configuration to be made within a limited time. When time is not severely constrained, the a priori choice of a computational budget becomes unjustifiable because the relationship between the computational budget and the quality of the optimization of a black box subject is not obvious. This paper proposes an extension to F-Race in the form of a heuristic method for reasonably terminating the optimization procedure.