A simple `linearized´ learning algorithm which outperforms back-propagation

A class of algorithms is presented for training multilayer perceptrons using purely linear techniques. The methods are based upon linearization of the network using error surface analysis, followed by a contemporary least-squares estimation procedure. Specific algorithms are presented for estimating weights node-wise and layer-wise and for estimating the entire set of network weights simultaneously. In several experimental studies, the node-wise method was superior to backpropagation and to an alternative linearization method due to M. Azimi-Sadjadi et al. (1990) in terms of number of convergences and convergence rate. The layer-wise and network-wise updating offer further improvement