Improving three layer neural net convergence

The authors investigate the relationship between three layer feed forward back-propagation nets (using the terminology of Rumelhart et al., see Nature vol.323, p.533 et seq., 1986) and the committee net of (Nilsson, see Learning Machines, McGraw-Hill, 1956), and show that a simple modification to the algorithm of the latter makes them, in respect of their power to classify data sets, equivalent. Two algorithms may, however, be equivalent in power but differ greatly in their practicality. In the second part the authors conduct some experiments in order to determine whether the modified committee algorithm can compete with back-propagation in a variety of applications. It is found that the committee algorithm (a) is about ten times as fast in some applications and (b) is much less prone to getting trapped in local minima. The theoretical interest in the paper stems from the ease of analysing the committee algorithm together with the equivalence. The experimental interest is that this method of speeding up back-propagation may be used with other improvements to reduce training times in some applications