Connections between optimisation-based regressor selection and analysis of variance

Earlier contributions have shown that analysis of variance (ANOVA) can be successfully used for finding good regressors for nonlinear models in a nonlinear black-box system identification context. In this paper, it is shown that the ANOVA problem can be recast as an optimisation problem. Two modified, convex versions of the ANOVA optimisation problem are then proposed, and it turns out that they are closely related to the nn-garrote and wavelet shrinkage methods, respectively. In the case of balanced data, it is also shown that the methods have a nice orthogonality property in the sense that different groups of parameters can be computed independently