Linearized least-squares training of multilayer feedforward neural networks

The authors develop a linearized least-squares formulation for estimating the weight coefficients of a neural network. Linearization of the nonlinear network about the most recent weight estimates leads to a conditional least-squares criterion which may be solved recursively in time. The resulting coefficient update equations resemble those of the recursive least-squares solution in adaptive filtering, much as the update equations for linearized stochastic gradient descent (backpropagation) resemble those of the least mean squares solution in adaptive filtering. Simulations on small logic mapping problems indicate a three- to tenfold increase in training efficiency for this technique as compared to gradient descent