Convergence Analysis of Generalized Back-propagation Algorithm with Modified Gradient Function

In this paper, we further investigate the convergence properties of the generalized back-propagation algorithm using magnified gradient function (MGFPROP). The idea of MGFPROP is to increase the convergence rate by magnifying the gradient function of the activation function. The algorithm can effectively speed up the convergence rate and reduce the chance of being trapped in premature saturation. From the convergence analysis, it is shown that MGFPROP retains the gradient descent property, gives faster convergence and has better global searching capability than that of the back-propagation algorithm.