A new training method for precision-limited analog neural networks

Neural networks with a large number of neurons can be implemented in hardware if the precision of the weights and inputs is limited. This paper centers on situations where two conditions are present; first, that the neural network is implemented using limited precision hardware, and second, that the weights were calculated using higher precision values than can be stored in the hardware. This paper shows that limited precision weights in the target system can result in forward calculation errors that are sufficiently large to cause classification mistakes in an example problem. The cause of classification mistakes is identified in terms of topological features in weight-space. A numerical calculation, that demonstrates the existence of weight-space regions where such errors occur, is presented. The Big Valley Search algorithm is proposed to overcome the effects of limited storage. Experimental results are presented to show that the Big Valley Search algorithm can be used to overcome implementation problems encountered when implementing a fault classifier using an analog neural network chip. Using experimental results, the requirement for chip-in-loop (CIL) programming of analog chips is shown to be unnecessary