Title :
New second-order algorithms for recurrent neural networks based on conjugate gradient
Author :
Campolucci, Paolo ; Simonetti, Michele ; Uncini, Aurelio ; Piazza, Francesco
Author_Institution :
Dipt. di Elettronica e Autom., Ancona Univ., Italy
Abstract :
We derive two second-order algorithms, based on the conjugate gradient method, for online training of recurrent neural networks. These algorithms use two different techniques to extract second-order information on the Hessian matrix without calculating or storing it and without making numerical approximations. Several simulation results for nonlinear system identification tests by locally recurrent neural networks are reported for both the off-line and online case
Keywords :
Hessian matrices; conjugate gradient methods; identification; learning (artificial intelligence); nonlinear systems; recurrent neural nets; Hessian matrix; conjugate gradient method; nonlinear system identification; off-line training; online training; recurrent neural networks; second-order algorithms; second-order information; Character generation; Convergence; Data mining; Digital signal processing; Electronic mail; IP networks; Iterative algorithms; Neural networks; Recurrent neural networks; System testing;
Conference_Titel :
Neural Networks Proceedings, 1998. IEEE World Congress on Computational Intelligence. The 1998 IEEE International Joint Conference on
Conference_Location :
Anchorage, AK
Print_ISBN :
0-7803-4859-1
DOI :
10.1109/IJCNN.1998.682297
