Davidon Least Squares based Neural Network Learning Algorithms

This paper presents a new learning methodology for feedforward neural networks. The proposed algorithm is based on Davidon´s Lea-square minimization approach. We modified the original Davidon algorithm so that it can handle more than one error term at each iteration. This increases the cost of computation but a trade-off can be achieved between the increase in cost and the improvement in accuracy. We compared the performance of Davidon algorithm with that of the backpropagation method while we trained a feedforward neural network. We also compared our modified Davidon algorithm with the original version of the Davidon algorithm. Our network is employed to predict the one step ahead output of a given dynamic system The simulation results show that based on the same stopping criteria, the Davidon algorithm is not necessarily faster than the backpropagation algorithm. However, the prediction error variance is significantly lower than that of backpropagation. Furthermore, the proposed modified Davidon algoritm performed better than the Davidon algorithm in predicting the system output with much higher. accuracy.