Nonlinear quantization effects in the frequency domain complex scalar LMS adaptive algorithm

Nonlinear quantization effects in the frequency domain complex scalar LMS adaptive algorithm are analyzed by using conditional expectations. The probability density function of the quantizer input, conditioned on the weight, is derived. The density is applied to finding the conditional characteristic function and the Mth conditional moment at the quantizer output. The first and second conditional moments of the quantizer output are used to derive difference equations that approximate the dynamical behavior of the first and second weight moments. These difference equations are solved numerically and compare favorably to simulation results. A model of the quantizer as an additive noise source is of no analytical value since the quantizatian noise has negligible effect on the mean square error when the model is valid. Finally, a design approach is proposed for selecting the number of bits in the weight accumulator. The moment equations are also used to determine the algorithm mean square error for different quantizer step sizes and the optimum algorithm step size μ when a fixed amount of input data is available for adaptation.