Improved second-order training algorithms for globally and partially recurrent neural networks

Recurrent neural networks are dynamic nonlinear systems that can exhibit a wide range of behaviors. However, the availability of recurrent neural networks of practical importance is associated with the existence of efficient supervised learning algorithms based on optimization procedures for adjusting the parameters. To improve performance, second order information should be considered to minimize the error in the training process. The first objective of this work is to describe systematic ways of obtaining exact second-order information for a range of recurrent neural network configurations, with a low computational cost. The second objective is to present an improved version of the conjugate gradient algorithm that can be used to effectively explore the available second-order information