Incremental communication for multilayer neural networks in a field programmable gate array

A neural network is a massively parallel distributed processor made up of simple processing units known as neurons. These neurons are organized in layers and every neuron in each layer is connected to each neuron in the adjacent layers. This connection architecture makes for an enormous number of communication links between neurons. This is an issue when considering a hardware implementation of a neural network since communication links requires costly hardware space. To overcome this space problem incremental communication for multilayer neural networks has been proposed. Incremental communication works by only communicating the change in value between neurons as opposed to the entire magnitude of the value. This allows for the values to be represented with a fewer number of bits, and thus communicated with narrower communication links. To validate and analyze this technique a neural network is designed and implemented using both an incremental and traditional communication approach.