A new local linearized least squares algorithm for training feedforward neural networks

An algorithm used to train the weights of a feedforward neural network is the global extended Kalman filter (GEKF) algorithm, which has much better performance than the popular gradient descent with error backpropagation in terms of convergence and quality of solution. However, the GEKF is very computationally intensive, and this has led to the development of simplified algorithms based on the partitioning of the global nonlinear optimization problem into a set of local nonlinear problems at the neuron level. In this paper a new training algorithm is developed by viewing the local subproblems as recursive linearized least squares problems. The objective function of the least squares problems for each neuron is the sum of the squares of the linearized backpropagated error signals. The new algorithm is shown to give better convergence results for two benchmark problems in comparison to existing local algorithms