Title :
On Descent Spectral CG Algorithms for Training Recurrent Neural Networks
Author :
Livieris, I.E. ; Sotiropoulos, D.G. ; Pintelas, P.
Author_Institution :
Dept. of Math., Univ. of Patras, Patras, Greece
Abstract :
In this paper, we evaluate the performance of descent conjugate gradient methods and we propose a new algorithm for training recurrent neural networks. The presented algorithm preserves the advantages of classical conjugate gradient methods while simultaneously avoids the usually inefficient restarts. Simulation results are also presented using three different recurrent neural network architectures in a variety of benchmarks.
Keywords :
conjugate gradient methods; learning (artificial intelligence); neural net architecture; recurrent neural nets; conjugate gradient methods; neural networks training; recurrent neural networks; spectral CG algorithms; Character generation; Computer networks; Convergence; Feedforward neural networks; Gradient methods; Informatics; Iterative algorithms; Mathematics; Neural networks; Recurrent neural networks; Recurrent neural networks; descent spectral conjugate gradient methods; performance profiles; sufficient descent property;
Conference_Titel :
Informatics, 2009. PCI '09. 13th Panhellenic Conference on
Conference_Location :
Corfu
Print_ISBN :
978-0-7695-3788-7
DOI :
10.1109/PCI.2009.33
