Creating Diversity in Ensembles using Clustering Method from Libraries of Models

The diversity of an ensemble of models is known to be an important factor in improving its generalization performance. We present an ensemble method based on clustering technique from libraries of models, EMC (ensemble of models based on clustering). It may be seen as a unified ensemble approach based on clustering. First, model libraries are generated using different learning algorithms and parameter settings, for example, neural networks (NNs), support vector machines (SVMs), and decision trees (DTs). Then clustering method is used to select the diverse models in ensembles. Finally, the selected partial models´ predictions are combined by voting. Experiments with 10 representative data sets from UCI repository demonstrate the benefit of ensemble selection