Analysis of support vector machines

We compare L1 and L2 soft margin support vector machines from the standpoint of positive definiteness, the number of support vectors, and uniqueness and degeneracy of solutions. Since the Hessian matrix of L2 SVM is positive definite, the number of support vectors for L2 SVM is larger than or equal to the number of L1 SVM. For L1 SVM, if there are plural irreducible sets of support vectors, the solution of the dual problem is non-unique although the primal problem is unique. Similar to L1 SVM, degenerate solutions, in which all the data are classified into one class, occur for L2 SVM.