Surrogate models in ill-conditioned systems

Ridge versions of an ill-conditioned system are alleged to “act more like an orthogonal system” than the system itself. Alternatives, called surrogates and based on the conditioning of linear systems, are shown to yield smaller expected mean squares than OLS, and uniformly smaller residual sums of squares than ridge. Ridge and surrogate solutions are compared on several marques of orthogonality to include conditioning of dispersion arrays, variance inflation factors, isotropy of variances, and sphericity of contours of the estimators. For these, ridge typically exhibits erratic divergence from orthogonality as the ridge scalar evolves, often reverting back to OLS in the limit. In contrast, surrogate solutions converge monotonically to those from orthogonal systems. Invariance considerations constrain the computations to models in canonical form. Case studies serve to illustrate the central issues.