Training a one-dimentional classifier to minimize the probability of error

We report some of the results of a study of asymptotically optimum nonparametric training procedures for two-category pattern classifiers. The decision surfaces yielded by earlier forms of nonparametric training procedures generally do not minimize the probability of error. We derive a modification of the Robbins-Monro method of stochastic approximation, and show how this modification leads to training procedures that minimize the probability of error of a one-dimensional two-category pattern classifier. The class of probability density functions admitted by these training procedures is quite broad, permitting a combination of continuous and discrete components in the density functions. We show that the sequence of decision points generated by any of these training procedures converges with probability one to the minimum-probability-of-error decision point.