Accurate Estimate of the Cross-Validated Prediction Error Variance in Bayes Classifiers

A relationship between the variance of the prediction error committed by the Bayes classifier and the mean prediction error was established by experiments in emotional speech classification within a cross-validation framework in a previous work. This paper theoretically justifies the validity of the aforementioned relationship. Furthermore, it proves that the new estimate of the variance of the prediction error, treated as a random variable itself, exhibits a much smaller variance than the usual estimate obtained by cross- validation even for a small number of repetitions. Accordingly, we claim that the proposed estimate is more accurate than the usual, straightforward, estimate of the variance of the prediction error obtained by applying cross-validation.