Backpropagation topologies for sequence generation

The problem of generating the correct spelling of an incorrectly spelled name was used to explore the effectiveness of various backpropagation network topologies for sequential generation. Two sequential architectures, a Jordan net and a counter net, learned much more slowly than a standard parallel net. Best results were obtained when the task was decoupled into two separate nets, one to generate unordered letters and another to reorder the letters. The first net was trained independently, and the second net was trained by recoupling the two nets so that the output of the first served as input to the second.<>