Multistage classification by cascaded classifiers

We propose a new method of classification built as a cascade of a distributed learner and a local learner. The distributed learner generalizes to learn the “rule” and the local learner learns the “exceptions” not covered by the “rule”. We show how such a system can be trained using cross-validation. We use a multilayer perceptron with sigmoidal hidden units as the rule-learner and a k-nearest neighbor classifier as the exception-learner. Cascading is a better approach than voting where multiple learners are used for all cases; the extra computation and memory required for the second learner is unnecessary if we are sufficiently sure that the first one´s response is correct. The cascade algorithm significantly outperforms the individual methods and voting on three optical and pen-based handwritten digit recognition tasks when comparison is based on three criteria; generalization success, learning speed, and number of free parameters