Multi-layer perceptrons with discrete weights

The feasibility of restricting the weight values in multilayer perceptrons to powers of two or sums of powers of two is studied. Multipliers could be thus replaced by shifters and adders on digital hardware, saving both time and chip area, under the assumption that the neuron activation function is computed through a lookup table (LUT) and that a LUT can be shared among many neurons. A learning procedure based on back-propagation for obtaining a neural network with such discrete weights is presented. This learning procedure requires full real arithmetic and therefore must be performed offline. It starts from a multilayer perceptron with continuous weights learned using back-propagation. Then a weight normalization is made to ensure that the whole shifting dynamics is used and to maximize the match between continuous and discrete weights of neurons sharing the same LUT. Finally, a discrete version of BP algorithm with automatic learning rate control is applied up to convergence. Some test runs on a simple pattern recognition problem show the feasibility of the approach