On the theory for autoregressive processes

A theoretical framework for autoregressive estimation is presented. Three levels of approximation are distinguished: probability limits, asymptotic theory and finite sample theory. At each level, formulae are given for the variance of estimated parameters, for the residual variance which is minimized and for the prediction error which is a measure for the accuracy of a model. The probability limits provide no grounds for order selection, because all models above the true process order are equal at this level. The asymptotic theory yields FPE, AIC and related consistent criteria for order selection. The finite sample theory takes into account the differences that have been observed between various estimation methods and the dependence on the model order