Boosting for learning from multiclass data sets via a regularized loss function

Boosting methods employ a sequence of base classifiers to improve accuracy. While successful with binary classification, the conversion from binary boosting to multi-class boosting is not straight forward. The direct extension avoids converting the problem into multiple binary problems but suffers several problems including a vague determination of the error condition for each base classifier, an elongated training time that increases significantly with large datasets, early termination which results from repetition of misclassified instances, and inability to employ stable classifiers within the boosting scheme. In this paper, we introduce a direct multi-class boosting extension that combines an intelligent class-based stratified sampling with a regularization parameter that regularizes the unpredictability of the classifiers and accommodates multi-class data sets. The parameter extends the exponential loss function and penalizes the base classifier when it favors specific instances on expense of the correct distribution. This methodology ensures diversity among classifiers which is vital to ensemble learning. The proposed method alters the error condition at each iteration based on the base classifiers performance. Experimental results on different applications demonstrate the superior performance of our method compared to state-of-the-art methods.