Back-propagation algorithm with controlled oscillation of weights

Error back propagation (EBP) is a training algorithm for feedforward artificial neural networks (FFANNs). Certain simple but effective modifications to the EBP algorithm to improve its convergence are proposed. The methods watch for oscillation of weights as the training algorithm proceeds. When such an oscillation is observed, the learning rate for only that weight is temporarily reduced. Effective methods are found for temporary reduction of learning rate for individual oscillating weight. To study the performance of the authors´ modified learning algorithm, extensive simulation with one complex learning problem is performed. The simulation results favorably compare the modified algorithms. The modified EBP algorithms, like the EBP algorithm, monotonically reduce the sum-of-the-squared error as the training proceeds