Multi-layer perceptron ensembles for pattern recognition: some experiments

All global optimization methods trying to select the best model for a given task have to face the problem of the bias introduced during learning by several factors (e.g., overfitting problems or local minima) which may lead to poor generalization. Multiple models can reduce this problem by combining the results of a population of differently trained networks. Unfortunately, nonlinear adaptive systems like multilayer perceptrons cannot be integrated in the parameter (weight) space. Therefore, a key issue in the development of efficient network ensembles is the trade-off between performance improvement and the required resource overhead with respect to standard optimal model selection techniques. In this work several approaches to the design of multiple multilayer perceptrons classifiers are compared, and their robustness to some implementation constraints is evaluated