Why pairwise is better than one-against-all or all-at-once

In this paper, first we discuss acceleration of classification by reducing support vectors. Then, we discuss multiclass least squares SVMs (LS-SVMs) that resolve unclassifiable regions for multiclass problems: fuzzy one-against-all LS-SVMs, fuzzy pairwise LS-SVMs, and all-at-once LS-SVMs. Next, we compare the three types of LS-SVMs from the standpoint of training difficulty and show that the fuzzy one-against-all LS-SVM and the all-at-once LS-SVM have similar decision boundaries when classification problems are linearly separable in the feature space. Finally, we evaluate three types of multiclass LS-SVMs for some benchmark data sets and show that classification performance of fuzzy one-against-all and one-against-all LS-SVMs are almost the same but inferior to that of fuzzy pairwise LS-SVMs.