Tuning complexity in kernel-based linear system identification: The robustness of the marginal likelihood estimator

In recent works, a new regularized approach for linear system identification has been proposed. The estimator solves a regularized least squares problem and admits also a Bayesian interpretation with the impulse response modeled as a zero-mean Gaussian vector. A possible choice for the covariance is the so called stable spline kernel. It encodes information on smoothness and exponential stability, and contains just two unknown parameters that can be determined from data via marginal likelihood (ML) optimization. Experimental evidence has shown that this new approach may outperform traditional system identification approaches, such as PEM and subspace techniques. This paper provides some new insights on the effectiveness of the stable spline estimator equipped with ML for hyperparameter estimation. We discuss the mean squared error properties of ML without assuming the correctness of the Bayesian prior on the impulse response. Our findings reveal that many criticisms on ML robustness are not well founded. ML is instead valuable for tuning model complexity in linear system identification also when impulse response description is affected by undermodeling.