Dynamic range and finite word effects in digital implementation of the LMS algorithm

The effect of a 1-e^-x saturation type nonlinearity on the weight update in LMS (least-mean-squares) adaptation is investigated for a white Gaussian data model. Nonlinear difference equations are derived for the weight first and second moments. A nonlinear difference equation for the mean norm is explicitly solved by a differential equation approximation and integration by quadratures. The steady-state second-moment-weight behavior is evaluated approximately. Using these results, the tradeoff between the extent of weight update saturation, steady-state excess mean-square-error, and rate of algorithm convergence is studied. For the same steady-state misadjustment error, the tradeoff shows that: (1) starting with a sign detector, the convergence rate is increased by nearly a factor of two for each additional bit, and (2) as the number of bits is increased further, the additional bits buy very little in convergence speed, asymptotically approaching the behavior of the linear model