Derivation of momentum LMS algorithms by minimizing objective functions

Author

Dahanayake, B.W. ; Upton, A.R.M.

Author_Institution

Dept. of Med., McMaster Univ., Hamilton, Ont., Canada

fYear

1993

fDate

1993

Firstpage

831

Abstract

The momentum least-mean-squares (m-LMS) algorithm is extensively used in neural network and signal processing applications, and is an arbitrary extension to the LMS algorithm. It is shown that several different versions of the m-LMS algorithm can be obtained by minimizing different objective functions. It appears that the minimization of weighted average square error function and the weighted accumulated square error function leads to two widely used m-LMS algorithms. The minimization of the weighted average square error function also provides two new versions of the m-LMS algorithm. These old and new versions of the m-LMS algorithm are applied to a parameter estimation problem. From the results, it is found that the new versions of the m-LMS algorithm provide smaller variance of the parameter estimates

Keywords

least squares approximations; neural nets; signal processing; momentum LMS algorithms; momentum least-mean-squares algorithm; neural network; objective function minimization; parameter estimation; signal processing; weighted accumulated square error function; weighted average square error function; Biological neural networks; Biomedical signal processing; Convergence; Least squares approximation; Minimization methods; Nervous system; Parameter estimation; Signal processing algorithms; Vectors;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks, 1993., IEEE International Conference on

Conference_Location

San Francisco, CA

Print_ISBN

0-7803-0999-5

Type

conf

DOI

10.1109/ICNN.1993.298666

Filename

298666