A generalization technique for nearest-neighbor classifiers

The authors present a novel algorithm which provides nearly optimal generalization capabilities to k-NN (nearest neighbor) classifiers. It prunes the training set and, at the same time, improves the performance of the classifier. Its computational cost is a weak function of the dimensionality of the space of the training set and it is not related to any specific formulation of the metric used in the classifier. The asymptotic computational cost of the proposed algorithm is a low-order polynomial of the number of elements in the training set. It is also proved that, given a suitable performance measure, each set of the generalization algorithm monotonically improves the performances of the classifier. Finally, experimentally results stemming from the classification of handwritten digits show that the training set provided to the classifier can be reduced by more than one order of magnitude, thus dramatically reducing the CPU time and the memory required to classify new samples