Stein´s paradox and improved quadratic discrimination of real and simulated data by covariance weighting

The cross-validation error rates of quadratic discriminant classifiers were substantially reduced by weighting the off-diagonal elements of the covariance matrices by a constant less than one. The improvement was found both when real chromosome data was classified, and also when simulated multivariate normal random vectors were classified. It is shown empirically that the optimum value of the weighting constant depended on the size of the training set, and that its relative benefit was greatest when the training set was small. The optimum weight appeared to be largely independent of the number of features. The relationship of this heuristic to Stein´s paradox (1956) was explored, and it is shown that near-optimal values of weight could be predicted directly, thereby avoiding the need for an expensive empirical search