Support vector regression performance analysis and systematic parameter selection

Support vector regression (SVR) based on statistical learning is a useful tool for nonlinear regression problems. The SVR method deals with data in a high dimension space by using linear quadratic programming techniques. As a consequence, the regression result has optimal properties. However, if parameters were not properly selected, overfitting and/or underfilling phenomena might occur in SVR. Two parameters σ, the width of Gaussian kernels and ε, the tolerance zone in the cost function are considered in this research. We adopted the concept of the sampling theory into Gaussian filter to deal with parameter σ. The idea is to analyze the frequency spectrum of training data and to select a cut-off frequency by including 90% of power in spectrum. The corresponding σ can then be obtained through the sampling theory. In our simulations, it can be found that good performances are observed when the selected frequency is near the cut-off frequency. For another parameter ε, it is a tradeoff between the number of support vectors and the RMSE. By introducing the confidence interval concept, a suitable selection of ε can be obtained. The idea is to use the L₁-norm (i.e., when ε = 0 ) to estimate the noise distribution of training data. When ε is obtained by selecting the 90% confidence interval, simulations demonstrated superior performance in our illustrative example. By our systematical design, proper values of σ and ε can be obtained and the resultant system performances are nice in all aspects.