Optimal subspace metric design for classification and regression through nonparametric cross validation objective function optimization

An approach for kernel design is presented. The technique is called “optimal subspace metric design” (OSMD). One of the main drawbacks of many distance based parametric and nonparametric approaches is that they do not use a metric which has a certain kind of orientation, which suits best for the given application. In the approach presented a kernel based technique with a full covariance metric is used. The need for an optimal, task dependent covariance structure is important for the mapping performance. A technique based on Householder reflections is derived, which allows us to model the covariance structure in an optimization procedure. The number of parameters used is only linearly dependent on the input dimension. It is possible to model a singular metric, to obtain a subspace approach. Both the orientation and the kernel width´s of the metric are adapted. To optimize the metric an error function based on the leave one out cross-validation technique is derived. The model is trained in a supervised manner, via the extended Kalman filter technique, and is able to handle both classification and regression tasks