Comparison between steepest descent and LMS algorithms in adaptive filters

It is commonly stated that the least-mean-square (LMS) algorithm for adaptive filters is a stochastic version of the steepest descent (SD) optimisation technique, although little work on comparative studies has been reported. The present paper sets out a detailed theoretical and experimental comparison. Equations are derived for the directional variance of the estimated gradient, and these are then experimentally verified by means of a constrained LMS simulation¿¿an ensemble of LMS gradients is computed for a set of points determined by advancing an adaptive system according to the SD gradient. Particular attention is focused on the convergence process, since the LMS algorithm has been criticised for being too slow.