DocumentCode :
1763768
Title :
Clustering-Based Ensembles as an Alternative to Stacking
Author :
Jurek, Anna ; Yaxin Bi ; Shengli Wu ; Nugent, Chris D.
Author_Institution :
Sch. of Comput. & Math., Univ. of Ulster, Newtownabbey, UK
Volume :
26
Issue :
9
fYear :
2014
fDate :
Sept. 2014
Firstpage :
2120
Lastpage :
2137
Abstract :
One of the most popular techniques of generating classifier ensembles is known as stacking which is based on a meta-learning approach. In this paper, we introduce an alternative method to stacking which is based on cluster analysis. Similar to stacking, instances from a validation set are initially classified by all base classifiers. The output of each classifier is subsequently considered as a new attribute of the instance. Following this, a validation set is divided into clusters according to the new attributes and a small subset of the original attributes of the instances. For each cluster, we find its centroid and calculate its class label. The collection of centroids is considered as a meta-classifier. Experimental results show that the new method outperformed all benchmark methods, namely Majority Voting, Stacking J48, Stacking LR, AdaBoost J48, and Random Forest, in 12 out of 22 data sets. The proposed method has two advantageous properties: it is very robust to relatively small training sets and it can be applied in semi-supervised learning problems. We provide a theoretical investigation regarding the proposed method. This demonstrates that for the method to be successful, the base classifiers applied in the ensemble should have greater than 50% accuracy levels.
Keywords :
learning (artificial intelligence); pattern classification; pattern clustering; base classifiers; class label; clustering-based ensembles; meta-classifier; meta-learning approach; semisupervised learning problems; validation set; Accuracy; Mathematical model; Probability distribution; Semisupervised learning; Stacking; Training; Clustering; Combining Classifiers; Combining classifiers; Ensembles; Meta-Learning; Semi-Supervised Classification; Stacking; clustering; ensembles; meta-learning; semi-supervised classification; stacking;
fLanguage :
English
Journal_Title :
Knowledge and Data Engineering, IEEE Transactions on
Publisher :
ieee
ISSN :
1041-4347
Type :
jour
DOI :
10.1109/TKDE.2013.49
Filename :
6482565
Link To Document :
بازگشت