Circuit implementation of a trainable neural network using the generalized Hebbian algorithm with supervised techniques

An efficient hardware implementation of a training algorithm is presented, combining both supervised and unsupervised techniques. In VLSI circuits effects including random offsets and mismatch, system distortion, frequency response, and temperature deviations perturb the system outputs. Analysis of the generalized Hebbian algorithm (GHA) shows that these small deviations result in small perturbations of the output statistics and the weight matrix. Also, the conjugate-gradient optimization algorithm is formulated in continuous-time. Both the GHA and the optimization system can be efficiently implemented on a mesh of synapses. The low distortion and high bandwidth (>100 MHz) for the matrix operations and a high-performance analog memory indicate high performance for the GHA. From the mathematical theory, the error of the output statistics is less than 1%