On the convergence of least squares estimates in white noise

The problem of convergence of least squares (LS) estimates in a stochastic linear regression model with white noise is considered. It is well known that if the parameter estimates are known to converge, the convergence analysis for many adaptive systems can be rendered considerably less arduous. For an important case where the regression vector is a measurable function of the observations and the noise is Gaussian, it has been shown, by using a Bayesian embedding argument, that the LS estimates converge almost surely for almost all true parameters in the parameter space except for a zero-measure set. However, nothing can be said about a particular given system, which is usually the objective. It has long been conjectured that such a “bad” zero measure set in the parameter space does not actually exist. A conclusive answer to this important question is provided and it is shown that the set can indeed exist. This then shows that to provide conclusive convergence results for stochastic adaptive systems, it is necessary to resort to a sample pathwise analysis instead of the Bayesian embedding approach