Abstract :
We introduce basic ideas of binary separation by a linear hyperplane, which
is a technique exploited in the support vector machine (SVM) concept. This
is a decision-making tool for pattern recognition and related problems. We
describe a fundamental standard problem (SP) and show how this is used in
most existing research to develop a dual-based algorithm for its solution. This
algorithm is shown to be deficient in certain aspects, and we develop a new
primal-based SQP-like algorithm, which has some interesting features. Most
practical SVM problems are not adequately handled by a linear hyperplane.
We describe the nonlinear SVM technique, which enables a nonlinear separating
surface to be computed, and we propose a new primal algorithm based
on the use of low-rank Cholesky factors.
It may be, however, that exact separation is not desirable due to the presence
of uncertain or mislabelled data. Dealing with this situation is the main
challenge in developing suitable algorithms. Existing dual-based algorithms
use the idea of L1 penalties, which has merit. We suggest how penalties can be
incorporated into a primal-based algorithm. Another aspect of practical SVM
problems is often the huge size of the data set, which poses severe challenges
both for software package development and for control of ill-conditioning.
We illustrate some of these issues with numerical experiments on a range of
problems.
