Circuit implementation of trainable neural networks employing both supervised and unsupervised techniques

An efficient hardware implementation of a training algorithm is presented, combining supervised and unsupervised techniques. In VLSI circuits, effects including random offsets and mismatch, system distortion, frequency response, and temperature deviations perturb the system outputs. Analysis of the generalized Hebbian algorithm shows that these small deviations result in small perturbations of the output statistics and the weight matrix. Also, the conjugate-gradient optimization algorithm is formulated in continuous-time. Both the generalized Hebbian algorithm and the optimization system can be efficiently implemented on a mesh of synapses. The low distortion and high bandwidth (>100 MHz) for the matrix operations and a high-performance analog memory indicate high performance for the generalized Hebbian algorithm. From the mathematical theory, the error of the output statistics is less than 1%