Performance of the ridge regression method as applied to complex linear and nonlinear models

Three practical cases for application of ridge regression are studied and illustrated through mathematical derivation and computer simulation. The first case is the multiple linear model with parameters of similar magnitude, where the choice between the correlation form and original form arose. It is shown that the use of correlation form (centering and scaling the variables) gives worse parameter estimates than the use of the original form, thus keeping the original variables is advisable. The second case is the linear model with parameters of different order of magnitude. The conclusion is that this difference should be considered as far as possible by appropriate scaling during parameter estimation. The third case is a nonlinear model, where the method is also applicable. In all three cases, the improvement to the ordinary least squares (OLS) method is tremendous, in both the relevant mean square error function and the ridge trace.