MMI training of minimum complexity adaptive nearest neighbor classifiers

In this paper a minimum complexity adaptive nearest neighbor classifier “ANNC” is proposed, with maximum mutual information “MMI” training. The ANNC employs a winner-Gaussian approximation for each class PDF, with radially symmetrical and equal width Gaussians to produce piece-wise linear decision boundaries between classes. The MMI training minimizes an upper bound of the classification error probability, and thus is used to estimate the ANNC parameters. A discrete stochastic complexity criterion for classification “DSCC” is derived from the Bayesian model selection framework to estimate the minimum number of Gaussians required by the ANNC for optimal classification. Results of 3 experiments show the advantages of using the ANNC framework