Stochastic analysis of gradient adaptive identification of nonlinear systems with memory for Gaussian data and noisy input and output measurements

This paper investigates the statistical behavior of two gradient search adaptive algorithms for identifying an unknown nonlinear system comprised of a discrete-time linear system H followed by a zero-memory nonlinearity g(·). The input and output of the unknown system are corrupted by additive independent noises. Gaussian models are used for all inputs. Two competing adaptation schemes are analyzed. The first is a sequential adaptation scheme where the LMS algorithm is first used to estimate the linear portion of the unknown system. The LMS algorithm is able to identify the linear portion of the unknown system to within a scale factor. The weights are then frozen at the end of the first adaptation phase. Recursions are derived for the mean and fluctuation behavior of the LMS algorithm, which are in excellent agreement with Monte Carlo simulations. When the nonlinearity is modeled by a scaled error function, the second part of the sequential gradient identification scheme is shown to correctly learn the scale factor and the error function scale factor. Mean recursions for the scale factors show good agreement with Monte Carlo simulations. For slow learning, the stationary points of the gradient algorithm closely agree with the stationary points of the theoretical recursions. The second adaptive scheme simultaneously learns both the linear and nonlinear portions of the unknown channel. The mean recursions for the linear and nonlinear portions show good agreement with Monte Carlo simulations for slow learning. The stationary points of the gradient algorithm also agree with the stationary points of the theoretical recursions