Multidimensional splines with infinite number of knots as SVM kernels

Radial basis function (RBF) kernels for SVM have been routinely used in a wide range of classification problems, delivering consistently good performance for those problems where the kernel computations are numerically feasible (high-dimensional problems typically use linear kernels). One of the drawbacks of RBF kernels is the necessity of selecting the proper value of the hyperparameter γ in addition to the standard SVM penalty parameter C - this process can lead to overfitting. Another (more obscure) drawback of RBF is its inherent non-optimality as an approximation function. In order to address these issues, we propose to extend the concept of polynomial splines (designed explicitly for approximation purposes) to multidimensional normalized splines with infinite number of knots and use the resulting hyperparameter-free kernel SVMs instead of RBF kernel SVMs. We tested our approach for a number of standard classification datasets used in the literature. The results suggest that new kernels deliver mostly better classification performance than RBF kernel (for problems of moderately large dimensions), but allow faster computation (if measured on large cross-validation grids), with less chance of overfitting.