The generalized back-propagation algorithm with convergence analysis

The conventional back-propagation algorithm is basically a gradient-descent method, it has the problems of local minima and slow convergence. The generalized back-propagation algorithm which can effectively speed up the convergence rate has been proposed previously. In this paper, the convergence proof of the algorithm is analyzed. The generalized backpropagation algorithm changes the derivative of the activation function so as to magnify the backward propagated error signal when the output approaches a wrong value; thus the convergence rate can be accelerated and the local minimum escaped. From the convergence analysis, it is shown that the generalized back-propagation algorithm will improve the original backpropagation algorithm in terms of faster convergence and global search capability