DocumentCode :
1499220
Title :
Laplacian Embedded Regression for Scalable Manifold Regularization
Author :
Lin Chen ; Tsang, I.W. ; Dong Xu
Author_Institution :
Sch. of Comput. Eng., Nanyang Technol. Univ., Singapore, Singapore
Volume :
23
Issue :
6
fYear :
2012
fDate :
6/1/2012 12:00:00 AM
Firstpage :
902
Lastpage :
915
Abstract :
Semi-supervised learning (SSL), as a powerful tool to learn from a limited number of labeled data and a large number of unlabeled data, has been attracting increasing attention in the machine learning community. In particular, the manifold regularization framework has laid solid theoretical foundations for a large family of SSL algorithms, such as Laplacian support vector machine (LapSVM) and Laplacian regularized least squares (LapRLS). However, most of these algorithms are limited to small scale problems due to the high computational cost of the matrix inversion operation involved in the optimization problem. In this paper, we propose a novel framework called Laplacian embedded regression by introducing an intermediate decision variable into the manifold regularization framework. By using ∈-insensitive loss, we obtain the Laplacian embedded support vector regression (LapESVR) algorithm, which inherits the sparse solution from SVR. Also, we derive Laplacian embedded RLS (LapERLS) corresponding to RLS under the proposed framework. Both LapESVR and LapERLS posses a simpler form of a transformed kernel, which is the summation of the original kernel and a graph kernel that captures the manifold structure. The benefits of the transformed kernel are two-fold: (1) we can deal with the original kernel matrix and the graph Laplacian matrix in the graph kernel separately and (2) if the graph Laplacian matrix is sparse, we only need to perform the inverse operation for a sparse matrix, which is much more efficient when compared with that for a dense one. Inspired by kernel principal component analysis, we further propose to project the introduced decision variable into a subspace spanned by a few eigenvectors of the graph Laplacian matrix in order to better reflect the data manifold, as well as accelerate the calculation of the graph kernel, allowing our methods to efficiently and effectively cope with large scale SSL problems. Extensive experiments on both toy and r- al world data sets show the effectiveness and scalability of the proposed framework.
Keywords :
data handling; eigenvalues and eigenfunctions; graph theory; learning (artificial intelligence); least squares approximations; matrix inversion; optimisation; principal component analysis; regression analysis; sparse matrices; support vector machines; ∈-insensitive loss; LapERLS; LapESVR algorithm; LapRLS; LapSVM; Laplacian embedded RLS; Laplacian embedded regression; Laplacian embedded support vector regression algorithm; Laplacian regularized least squares; Laplacian support vector machine; SSL algorithms; computational cost; data manifold; decision variable; eigenvectors; graph Laplacian matrix; graph kernel; kernel matrix; kernel principal component analysis; labeled data; large-scale SSL problems; machine learning; matrix inversion operation; optimization problem; scalable manifold regularization; semisupervised learning; small-scale problems; sparse matrix; transformed kernel; unlabeled data; Eigenvalues and eigenfunctions; Kernel; Laplace equations; Manifolds; Optimization; Sparse matrices; Vectors; Laplacian embedding; large scale semi-supervised learning; manifold regularization;
fLanguage :
English
Journal_Title :
Neural Networks and Learning Systems, IEEE Transactions on
Publisher :
ieee
ISSN :
2162-237X
Type :
jour
DOI :
10.1109/TNNLS.2012.2190420
Filename :
6186826
Link To Document :
بازگشت