An incremental parallel tangent learning algorithm for artificial neural networks

Ghorbani and Bhavasar (1993) proposed a modified backpropagation training algorithm using a deflecting gradient method-parallel tangent (Partan) gradient method-to accelerate convergence. We have also proposed the incremental inter-neuron communication method. In the incremental communication method, instead of communicating the full magnitude of the output value of a node, only the (reduced precision) increment or decrement to its previous value is sent on a communication link. In this paper, Partan is modified to incorporate incremental communication. The effects of the limited precision incremental communication method on the convergence behaviour and performance of the Partan backpropagation algorithm are investigated with the sonar problem discussed in literature. It is found through simulation that the communication costs during the training process decrease by a factor of 1.6. The application of the incremental reduced precision Partan method to larger networks is expected to result into even higher savings in communication costs