Cumulative negative transfer during successive training: analysis of a second sequential learning problem

Summary form only given, as follows. Adaptive networks can be easily trained to associate arbitrary input and output patterns. When subgroups of patterns (lists) are presented sequentially, however, a network tends to ´unlearn´ previously acquired associations while learning new associations. A second form of sequential learning problem is reported. Learning of each successive list of pattern pairs becomes progressively more difficult. Evidence for this cumulative negative transfer was obtained from simulations using backpropagation of errors to train multilayer networks. The cause of the problem appears to be the development of extreme weights during the learning of new lists. Unbounded weights may be a liability for the backpropagation algorithm.<>