Optimum error nonlinearities for LMS adaptation

An examination is made of the effect of a memoryless nonlinearity acting upon the error in LMS (least-mean-square) adaptation. Results for E[e^2K] minimization are extended to general nonlinear error adaptation, and equations expressing bounds on step size, time constants, and misadjustment are derived. A general performance factor expressing the improvement in misadjustment over standard LMS adaptation for a given convergence rate is presented. Using the calculus of variations, it is shown that the optimum nonlinearity to minimize misadjustment is -p´(x)/p(x), where p(x ) is the probability density function of the uncorrelated plant noise. Comparisons of this result with the Cramer-Rao bound indicate that choice of this nonlinearity yields an asymptotically optimal stochastic gradient algorithm. Simulations verify the result that the optimum nonlinearity for minimizing misadjustment for Laplacian plant noise is sgn(x), and a 3-dB improvement is obtained under these conditions