Accelerated learning in multilayer networks

An accelerated learning algorithm, adaptive backpropagation (ABP), is proposed for the supervised training of multilayer networks. The algorithm is based on the principle of forced dynamics for the error functional. The algorithm does not require use of any information from previous updates, while it requires knowledge of exactly the same error terms used in standard backpropagation. The numerical simulation results indicate that there are certain advantages to using ABP. The method is consistently about an order of magnitude faster than the standard backpropagation method, and also faster than such accelerated algorithms as quickprop. There is no added tuning parameter, other than the learning rate, to which ABP appears to be less sensitive. However, the drawbacks of jumpy behavior in the vicinity of the local minima and the inability to eventually reach the global minimum exist