Effective neural network pruning using cross-validation

This paper addresses the problem of finding neural networks with optimal topology such that their generalization capability is maximized. Our approach is to combine the use of a penalty function during network training and a subset of the training samples for cross-validation. The penalty is added to the error function so that the weights of network connections that are not useful have small magnitude. Such network connections can be pruned if the resulting accuracy of the network does not change beyond a preset level. Training samples in the cross-validation set are used to indicate when network pruning is terminated. Our results on 32 publicly available data sets show that the proposed method outperforms existing neural network and decision tree methods for classification.