Evolutionary random neural ensembles based on negative correlation learning

This paper proposes to incorporate bootstrap of data, random feature subspace and evolutionary algorithm with negative correlation learning to automatically design accurate and diverse ensembles. The algorithm utilizes both bootstrap of training data and random feature subspace techniques to generate an initial and diverse ensemble and evolves the ensemble with negative correlation learning. The idea of generating ensemble by simultaneous randomization of data and feature is to promote the diversity within the ensemble and encourage different individual NNs in the ensemble to learn different parts or aspects of the training data so that the ensemble can learn better the entire training data. Evolving the ensemble with negative correlation learning emphasizes not only the accuracy of individual NNs but also the cooperation among different individual NNs and thus improves the generalization. As a byproduct of bootstrap, out-of-bag (OOB) estimation, which can estimate the generalization performance without any extra data points, serves another benefit of this algorithm. The proposed algorithm is evaluated by several benchmark problems and in these cases the performance of our algorithm is better than the performance of other ensemble algorithms.