Kernel selection in linear system identification Part I: A Gaussian process perspective

In some recent works, an alternative nonparametric paradigm to linear model identification has been proposed, where the unknown system impulse response is interpreted as a realization of a Gaussian process. Its autocovariance belongs to the class of so-called stable spline kernels that incorporate the stability constraint. Within this class, the order of the kernel establishes the degree of smoothness of the system impulse response. In this work, first we prove that such statistical models can be derived through Maximum Entropy arguments. Then, we show that the kernel order can be learnt from data via an efficient computational scheme that maximizes the marginal likelihood with respect to only two hyperparameters. Numerical experiments, with data generated by output error models, show the advantages of the new nonparametric estimator over the classical PEM approach that adopts cross validation to perform model order selection. In Part II of the companion papers the same identification problem is addressed in a deterministic framework.