Statistical analysis of neural network modeling and identification of nonlinear systems with memory

The paper presents a statistical analysis of neural network modeling and identification of nonlinear systems with memory. The nonlinear system model is comprised of a discrete-time linear filter H followed by a zero-memory nonlinear function g(.). The system is corrupted by input and output independent Gaussian noise. The neural network is used to identify and model the unknown linear filter H and the unknown nonlinearity g(.). The network architecture is composed of a linear adaptive filter and a two-layer nonlinear neural network (with an arbitrary number of neurons). The network is trained using the backpropagation algorithm. The paper studies the MSE surface and the stationary points of the adaptive system. Recursions are derived for the mean transient behavior of the adaptive filter coefficients and neural network weights for slow learning. It is shown that the Wiener solution for the adaptive filter is a scaled version of the unknown filter H. Computer simulations show good agreement between theory and Monte Carlo estimations