Ensemble pruning algorithms for accelerated training

The error surface on which minimization is done in any feedforward neural network training algorithm is highly irregular, with multiple local minima having been observed empirically. In training schemes, this implies that several random initial points must be chosen, and the performance of the resulting trained neural network evaluated for each such choice, in order to obtain a well-trained network. However, training is computationally expensive, and often one may have a limit on the number of training cycles allowed during training, thereby making the total number of cycles required to find the best-trained net too large for this brute-force method to be practical. It is therefore desirable to find an algorithm which eliminates “bad” networks during training itself, without utilizing the full allowed number of training cycles, and in such a way as to minimize the average total training cycles. We present two such algorithms which are easy to implement