A Weighted Kernel PCA Formulation with Out-of-Sample Extensions for Spectral Clustering Methods

A new formulation to spectral clustering methods based on the weighted kernel principal component analysis is presented. This formulation fits in the Least Squares Support Vector Machines (LS-SVM) framework as a primal-dual interpretation in the context of constrained optimization problems. Starting from the LS-SVM formulation to kernel PCA, a weighted approach is derived. An advantage of this method is the possibility to apply the trained clustering model to out-of-sample (test) data points without using approximation techniques such as the Nystrom method. Links with some existing spectral clustering techniques are given, showing that these techniques are particular cases of weighted kernel PCA. Simulation results with toy and real-life data show improvements in terms of generalization to new samples.