Learning with hidden targets

A fast-converging learning technique for multiple-layer feedforward neural networks, called dynamic hidden target (DHT), is presented. This learning technique eliminates the need for output error signal backpropagation and uses simple least mean square (LMS) gradient-descent adaptation at all layers. The algorithm estimates (in a minimum-squared-error sense) dynamic hidden targets for corresponding hidden layers based on the net´s output targets and updated synaptic weights located between the output targets and hidden neurons. Several simulations were performed which show at least one order-of-magnitude speedup of the DHT learning compared to standard backpropagation learning. The authors also compare DHT learning and several other versions of backpropagation, including the recently developed quickprop learning. DHT learning is shown to converge at least as fast as quickprop for the XOR and the 10-5-10 encoder/decoder problem and its complement. The DHT algorithm is superior to quickprop in that it involves a smaller number of learning parameters, does not require the estimation of second-order derivatives, and is robust against learning rate variations