Tolerance to analog hardware of on-chip learning in backpropagation networks

In this paper we present results of simulations performed assuming both forward and backward computation are done on-chip using analog components. Aspects of analog hardware studied are component variability, limited voltage ranges, components (multipliers) that only approximate the computations in the backpropagation algorithm, and capacitive weight decay. It is shown that backpropagation networks can learn to compensate for all these shortcomings of analog circuits except for zero offsets, and the latter are correctable with minor circuit complications. Variability in multiplier gains is not a problem, and learning is still possible despite limited voltage ranges and function approximations. Fixed component variation from fabrication is shown to be less detrimental to learning than component variation due to noise. Weight decay is tolerable provided it is sufficiently small, which implies frequent refreshing by rehearsal on the training data or modest cooling of the circuits. The former approach allows for learning nonstationary problem sets