New second-order algorithms for recurrent neural networks based on conjugate gradient

Author

Campolucci, Paolo ; Simonetti, Michele ; Uncini, Aurelio ; Piazza, Francesco

Author_Institution

Dipt. di Elettronica e Autom., Ancona Univ., Italy

Volume

1

fYear

1998

fDate

4-8 May 1998

Firstpage

384

Abstract

We derive two second-order algorithms, based on the conjugate gradient method, for online training of recurrent neural networks. These algorithms use two different techniques to extract second-order information on the Hessian matrix without calculating or storing it and without making numerical approximations. Several simulation results for nonlinear system identification tests by locally recurrent neural networks are reported for both the off-line and online case

Keywords

Hessian matrices; conjugate gradient methods; identification; learning (artificial intelligence); nonlinear systems; recurrent neural nets; Hessian matrix; conjugate gradient method; nonlinear system identification; off-line training; online training; recurrent neural networks; second-order algorithms; second-order information; Character generation; Convergence; Data mining; Digital signal processing; Electronic mail; IP networks; Iterative algorithms; Neural networks; Recurrent neural networks; System testing;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks Proceedings, 1998. IEEE World Congress on Computational Intelligence. The 1998 IEEE International Joint Conference on

Conference_Location

Anchorage, AK

ISSN

1098-7576

Print_ISBN

0-7803-4859-1

Type

conf

DOI

10.1109/IJCNN.1998.682297

Filename

682297