Semi-supervised image classification with huberized Laplacian Support Vector Machines

Semi-supervised learning has recently demonstrated be successful in large scale learning for image classification tasks. Laplacian Support Vector Machines (LapSVM) is one of such approaches applied to this task. However, LapSVM uses a squared hinge loss function for the labeled examples, which is not twice differentiable and may penalize noisy labeled examples too much. Thus, the accuracy decreases when the training data contains outliers or the labeled data is heavily contaminated by noise. We propose to use a continuously differentiable loss function called Huber hinge loss, which gives a milder penalty than the squared hinge loss. Furthermore, we build on the primal formulation of LapSVM and use a preconditioned conjugate gradient method to make the approach more efficient. In this way the training time can be reduced but still a very accurate approximation of the original problem can be obtained. Detailed experimental results validate our proposed strategy for classification problems when the available training data is contaminated with label-noise.