On the optimum data nonlinearity in LMS adaptation

The effect of an arbitrary nonlinear operation on the data input to the weight update equation in the LMS adaptive algorithm is investigated for a Gaussian data model. Exact difference equations are derived for the weight first and second moments that include the effects of an arbitrary nonlinear operation on the data sequence. The difference equations are used to obtain expressions for the transient behavior of the mean-square error. The mean-square error is minimized over the choice of nonlinearity for a fixed transient behavior. The best choice of nonlinearity is shown exactly to be of the form (x/1 + bx²) when the input data are white. For an arbitrary data covariance matrix, the optimum nonlinearity is shown to be linear when the product of the algorithm step size and input power is much less than unity.