The LMS algorithm with momentum updating

Author

Shynk, John J. ; Roy, Sumit

Author_Institution

Dept. Electr. & Comput. Eng., California Univ., Santa Barbara, CA, USA

fYear

1988

fDate

7-9 June 1988

Firstpage

2651

Abstract

Several modifications of the well-known least-mean-square (LMS) algorithm have been proposed for improved operation. The authors analyze one such recent innovation that corresponds to the ordinary LMS algorithm with an additional momentum term, parameterized by the factor alpha . The analysis of convergence in the mean yields some novel behavior insofar that it leads to complex eigenvalues of the transition matrix for the mean weight vector. The convergence in the mean-square analysis demonstrates that instability will occur as alpha tends closer to 1, a result not predicted by the analysis of convergence in the mean.<>

Keywords

convergence of numerical methods; eigenvalues and eigenfunctions; least squares approximations; matrix algebra; LMS algorithm; complex eigenvalues; convergence; least mean square algorithm; mean weight vector; momentum updating; transition matrix; Adaptive algorithm; Algorithm design and analysis; Computational complexity; Contracts; Convergence; Eigenvalues and eigenfunctions; Least squares approximation; Multi-layer neural network; Neural networks; Technological innovation;

fLanguage

English

Publisher

ieee

Conference_Titel

Circuits and Systems, 1988., IEEE International Symposium on

Conference_Location

Espoo, Finland

Type

conf

DOI

10.1109/ISCAS.1988.15485

Filename

15485