Designing a Multilayer Feedforward Ensembles with Cross Validated Boosting Algorithm

In previous researches we have analysed some methods to create committees of multilayer feedforward networks trained with the Back Propagation algorithm. One of the most known methods that we have studied is Adaptive Boosting. In this paper we present a variation of this method called Crossboost. In this version of AdaBoost, we have used k-cross-fold validation over the whole learning set to generate an specific training set and validation set for each network of the committee. In the new method, the data set used to train the ith-network is selectively sampled from its specific training set, the sampling distribution is calculated over the whole learning set. The diversity of a committee generated with our method increases respect the original method because each network has its specific validation set. We have tested Adaboost and Crossboost with ten databases from the UCI repository. We have used the mean percentage of error reduction to compare both methods, the results show that Crossboost performs better.