Polynomial componentwise LS-SVM: Fast variable selection using low rank updates

This paper describes a Least Squares Support Vector Machines (LS-SVM) approach to estimate additive models as a sum of non-linear components. In particular, this work discusses the low rank matrix modifications for componentwise polynomial kernels, which allow the factors of the modified kernel-matrix to be directly updated. The main concept refers to the use of a valid explicit feature map for polynomial kernels in an additive setting. By exploiting the structure of such feature map the model parameters of the classification/regression problem can be easily modified and updated when new variables are added. Therefore, the low rank updates constitute an algorithmic tool to efficiently obtain the model parameters once the system has been altered in some minimal sense. Such strategy allows, for instance, the development of algorithms for sequential variable ranking in high dimensional settings, while non-linearity is provided by the polynomial feature map. Moreover relevant variables can be robustly ranked using the closed form of the leave-one-out (LOO) error estimator, obtained as a by-product of the low rank modifications.