Multiprocessor implementations of neural networks

Four methodologies were developed to explore the potential of using affordable multiprocessor computers to implement commercially available neural networks currently run on single processor systems. The methodologies-layer, cross-layer, pipeline, and hybrid-are based on a framework built around the concepts of program decomposition, load balancing, communication overhead, and process synchronization. The methods were tested on a bus-based multiprocessor running two backpropagation network simulators, and the simulation results are reported. The pipeline and hybrid methods exhibited speedups of 1.6 and 1.9 per processor, respectively, for networks ranging in size from six to 20000 connections. These results indicate that it is possible to increase network simulation speeds and network size capabilities significantly