Evaluation of classifier performance in descrete pattern recognition problem

We consider a problem of pattern classifier performance evaluation in case of learning sample of limited size and discrete space of variables. The principle of Bayesian averaging of recognition performance is used for the analysis. With use of this principle, we found the dependencies between sample size, complexity of variables space, and the mean and variance of the true error function. This gives us a possibility to evaluate the confidence bound for the true error. As an application of these results, we consider the problem of classification tree design and evaluation of its performance.