Analysis of gradient descent learning algorithms for multilayer feedforward neural networks

The authors investigate certain dynamical properties of gradient-type learning algorithms as they apply to multilayer feedforward neural networks. These properties are more related to the multilayer structure of the net than to the particular output units at the nodes. The analysis is carried out on a simplified deterministic gradient algorithm in two steps. First, a global analysis of an associated ordinary differential equation (ODE) is performed using LaSalle´s theory. Then, a local analysis of the gradient algorithm is performed by linearizing along a nominal ODE trajectory. A simple numerical example is given to illustrate the analysis