Regression using Gaussian Process manifold kernel dimensionality reduction

This paper addresses the problem of learning optimal regressors that maximally reduce the dimension of the input while preserving the information necessary to predict the target values. Recent solutions to the sufficient dimensionality reduction and its generalizations to kernel settings, such as the manifold kernel dimensionality reduction (mKDR), rely on iterative schemes, without convergence guarantees. We show how a globally optimal solution in closed form can be obtained by formulating a related problem in a setting reminiscent of Gaussian Process (GP) regression. We then propose a generalization of the solution to arbitrary input points. In a set of experiments on real signal processing problems we show that the proposed GPMKDR can achieve significant gains in accuracy of prediction as well as interpretability, compared to other dimension reduction and regression schemes.