Putting the simple recurrent network to the test

Elman´s simple recurrent network is subjected to systematic empirical analysis. The principal results are that the network, when applied to the prediction task, is: surprisingly robust to variations of its operational parameters over large ranges of values; dependent on the presence of structure in the temporal sequences, to the point where the context memory hinders the network´s learning in totally unstructured domains; and specialized strategies excepted, i.e., incapable of retaining information across embedded inputs exceeding lengths of 2 or 3 tokens