An evaluation of constructive algorithms for recurrent networks on multi-step-ahead prediction

We evaluate on several multi-step-ahead prediction problems two constructive algorithms for recurrent neural networks, which were initially developed for learning long-range dependencies in the data. We compare both algorithms to the standard back-propagation through time and to other methods applied to the same datasets. The two algorithms improve over the results obtained by the standard back-propagation through time on these datasets and perform significantly better when long-range dependencies play an important role. We also find that the local approaches keep their advantage when compared to our global method, but with the price of a much higher number of parameters.