Title :
On the statistical efficiency of LMS algorithms
Author_Institution :
Dept. of Electr. Eng., Stanford Univ., CA, USA
Abstract :
Summary form only given. This paper describes the statistical efficiency of LMS algorithms. In this work, two gradient descent adaptive algorithms are compared, the LMS algorithm and the LMS/Newton algorithm. LMS is simple and practical and is used in many applications worldwide. LMS/Newton is based on Newton´s method and the LMS algorithm. LMS/Newton is optimal in the least squares sense. It maximizes the quality of its adaptive solution while minimizing the use of training data. Many least squares adaptive algorithms have been devised over the years, but no other least squares algorithm can give better performance, on average, than LMS/Newton. Furthermore, LMS algorithm is related to the famous backpropagation algorithm used for training neural networks.
Keywords :
adaptive filters; backpropagation; gradient methods; least mean squares methods; neural nets; statistical analysis; LMS algorithm; LMS-Newton algorithm; adaptive filter; backpropagation algorithm; gradient descent adaptive algorithm; neural network training; statistical efficiency; Adaptive algorithm; Autocorrelation; Backpropagation algorithms; Eigenvalues and eigenfunctions; Least squares approximation; Least squares methods; Neural networks; Performance evaluation; Testing; Training data;
Conference_Titel :
Signals, Systems and Computers, 2004. Conference Record of the Thirty-Eighth Asilomar Conference on
Print_ISBN :
0-7803-8622-1
DOI :
10.1109/ACSSC.2004.1399099
