Connectionist nonlinear over-relaxation

The nonlinear successive overrelaxation (NLOR) approach is adapted to create a connectionist heteroassociative algorithm with proven convergence properties. The algorithm developed here is shown to be similar to the widely used generalized delta rule, which does not have proven convergence properties. The NLOR heteroassociative algorithm incorporates delays in the weight updates to simplify the preprocessing necessary to ensure convergence. Simultaneous weight update is the most frequently used approach to learning in connectionist learning algorithms, but the asynchronous weight update presently used is both computationally and biologically preferable