"On the probability density function of the LMS adaptive filter weights"

In this paper, the joint probability density function of the weight vector in LMS adaptation is studied for Gaussian data models. An exact expression is derived for the characteristic function of the weight vector at time n+1 conditioned on the weight vector at time n. The conditional characteristic function is expanded in a Taylor series and averaged over the unknown weight density to yield a first order partial differential-difference equation in the un-conditioned characteristic function of the weight vector. The equation is solved approximately for small values of the adaptation parameter. The weights are shown to be jointly Gaussian with time varying mean vector and covariance matrix given as the solution to well-known difference equations for the weight vector mean and covariance matrix.