Finite sample properties of linear model identification

In this paper, we consider the finite sample properties of prediction error methods using a quadratic criterion function for system identification. The problem we pose is: How many data points are required to guarantee with high probability that the expected value of the quadratic identification criterion is close to its empirical mean value? The sample sizes are obtained using risk minimization theory which provides uniform probabilistic bounds on the difference between the expected value of the squared prediction error and its empirical mean evaluated on a finite number of data points. The bounds are very general. No assumption is made about the true system belonging to the model class, and the noise sequence is not assumed to be uniformly bounded. Further analysis shows that in order to maintain a given bound on the deviation, the number of data points needed grows no faster than quadratically with the number of parameters for FIR and ARX models