Large Scale Classification with Support Vector Machine Algorithms

Boosting of least-squares support vector machine (LS-SVM) algorithms can classify large datasets on standard personal computers (PCs). We extend the LS-SVM proposed by Suykens and Vandewalle in several ways to efficiently classify large datasets. We developed a row-incremental version for datasets with billions of data points and up to 10,000 dimensions. By adding a Tikhonov regularization term and using the Sherman-Morrison-Woodbury formula, we developed a column-incremental LS-SVM to process datasets with a small number of data points but very high dimensionality. Finally, by applying boosting to these incremental LS-SVM algorithms, we developed classification algorithms for massive, very-high-dimensional datasets, and we also applied these ideas to build boosting of other efficient SVM algorithms proposed by Mangasarian, including Lagrange SVM (LSVM), proximal SVM (PSVM) and Newton SVM (NSVM). Numerical test results on UCI, RCV1- binary, Reuters-21578, Forest cover type and KDD cup 1999 datasets showed that our algorithms are often significantly faster and/or more accurate than state-of- the-art algorithms LibSVM, SVM-perf and CB-SVM.