Solving flat-spot problem in back-propagation learning algorithm based on magnified error

A new learning algorithm based on magnified error is proposed to speedup the training of back-propagation neural networks, and to improve the performances of neural network. The key to this algorithm lies in varying the error item of output layer, which magnify the backward propagated error signal especially when the weight adjustment of output layer is slow or even suppressed. Therefore, the algorithm is able to get rid of the influence of "flat spot" problem, and solve the slow convergence problem. Consequently the convergence rate can be accelerated, and the training has great capability in meeting the convergence criteria quickly with a simple network structure. Experiments on parity-3 problem and soybean data classification problem show that this method has advantages of faster learning speed and less computational cost than most of the improved algorithms such as sigmoid-prime offset technique (SPO), scaled linear approximation of sigmoid method (SLA) and so on.