Impact of regularization in FIR estimation for short and long data records

The estimation of the impulse response of a linear dynamic system is of crucial importance in many measurement problems. When the task of collecting a large amount of measurements represents an expensive and time-consuming procedure, an accurate estimate needs to be extracted based on a short input/output data record. Well-tuned regularization methods are getting popular to improve the impulse response estimates in this and other situations, by reducing the model variance. Although it is commonly believed that the beneficial impact of regularization is mainly evident for short data records, in this paper it will be shown that this is also the case when a large amount of data is available. This surprising result is illustrated by Monte Carlo simulations comparing regularization and standard least squares.