A comparative study of fully and partially recurrent networks

A number of fully and partially recurrent networks have been proposed to deal with temporally extended tasks. However, it is not yet clear which algorithms and network architectures are best suited to certain kinds of problems. In this paper we report on experimental investigations of a quantitative nature, which address this particular need by comparing fully recurrent networks using learning algorithms such as backpropagation-through-time (BPTT), batch BPTT Quickprop-through-time, and real-time recurrent learning with Elman and Jordan partially recurrent networks on four benchmark problems: detection of three consecutive zeros, nonlinear plant identification, Turing machine emulation, and real-world distillation column modelling