Extending kernel principal component analysis to general underlying loss functions

Kernel principal component analysis can be considered as a natural nonlinear generalization of PCA because it performs linear PCA in a kernel induced feature space. It allows us to extract nonlinear structures in the input data. The classical kernel PCA formulation leads to an eigendecomposition of the kernel matrix: eigenvectors with large eigenvalue correspond to the principal components in the feature space. Starting from the least squares support vector machine (LS-SVM) formulation to kernel PCA we extend it to general underlying loss functions. For classical kernel PCA, the underlying loss function is L₂. In this approach, one can easily plug in other loss functions and solve a nonlinear optimization problem to achieve desirable properties. Simulations with Huber´s loss function for robustness and quadratic epsilon insensitive loss function for sparseness demonstrate the flexibility of our approach.