Tracking properties of adaptive signal processing algorithms

Adaptive signal processing algorithms are often used in order to "track" an unknown time-varying parameter vector. Such algorithms are typically some form of stochastic gradient-descent algorithm. The Widrow LMS algorithm is apparently the most frequently used. This work develops an upper bound on the norm-squared error between the parameter vector being tracked and the value obtained by the algorithm. The upper bound illustrates the relationship between the algorithm step-size and the maximum rate of variation in the parameter vector. Finally, some simple covariance decay-rate conditions are imposed to obtain a bound on the mean square error.