Analytic Study of Performance of Error Estimators for Linear Discriminant Analysis

We derive double asymptotic analytical expressions for the first moments, second moments, and cross-moments with the actual error for the resubstitution and leave-one-out error estimators in the case of linear discriminant analysis in the multivariate Gaussian model under the assumption of a common known covariance matrix and a fixed Mahalanobis distance as dimensionality approaches infinity. Sample sizes for the two classes need not be the same; they are only assumed to reach a fixed, but arbitrary, asymptotic ratio with the dimensionality. From the asymptotic moment representations, we directly obtain double asymptotic expressions for the bias, variance, and RMS of the error estimators. The asymptotic expressions presented here generally provide good small sample approximations, as demonstrated via numerical experiments. The applicability of the theoretical results is illustrated by finding the minimum sample size to bound the RMS in gene-expression classification.