Data-parallel training of spatiotemporal connectionist networks on the Connection Machine

An algorithm for optimizing spatiotemporal connectionist networks utilizing training set parallelism has been implemented on the Connection Machine (CM). The algorithm supports several optimization methods including backpropagation, conjugate gradient, and pseudo-Newtonian. By allocating one CM processor per training example, the computational complexity of the gradient derivation becomes independent of the number of training examples. The author has experimentally corroborated this independence, and reports the timing performance of the Connection Machine implementation on a series of spatiotemporal discrimination tasks. He also presents the timing performance of a serial implementation of the algorithm, running on an IBM RS/6000, to emphasize the efficacy of the data-parallel approach