Simple Algorithms for Least Square Support Vector Machines

In this paper, we propose five simple algorithms for solving the least square support vector machine (LS-SVM) learning problems. For linear regression, we first present a Widrow-Hoff-like algorithm for the primal optimization problem. The dual form of this algorithm is then provided. For kernel-based nonlinear LS-SVM, we first present a Widrow-Hoff-like algorithm. The elegant and powerful two-parameter sequential minimization optimization (2P-SMO) algorithm is then provided. Finally, we give a detailed derivation of the three-parameter sequential minimization optimization (3P-SMO) algorithm. A numerical example is provided.