A quasi-PID backpropagation algorithm based on Lyapunov stability theory for neural network

In this paper a new Lyapunov based backpropagation (BP) algorithm is proposed. The original BP algorithm is consists of one part, the learning rate factor (LR). In this new algorithm two extra adaptive parts has been added in comparison to original BP. The idea of adding these two parts is originated from the conventional PID controller. The first and second parts in the proposed algorithm are derivative and integral terms, respectively. These two parts solve two major limitations of the original BP algorithm, which are low speed and local minimum problem. As the stability is based on Lyapunov theory, the algorithm convergence is guaranteed. Finally the effectiveness of the proposed algorithm is evaluated via two examples XOR and 3-bit parity, and the results were compared with original BP. The results show the capability of the proposed algorithm in speed and the ability to escape from local minima.