Learning and generalization by coupled local minimizers

This paper introduces a fundamentally new method of coupled local minimizers. We show how state synchronization of continuous local optimization methods (coupled backpropagation learning processes in this case) can lead to cooperative search and improved solutions. We explain under which conditions the method leads to good generalization when applied to the training of MLPs, without using a regularization term in the cost function. The choice of the initial states of the minimizers plays an important role at this point. In the formulation, one takes identical copies of the cost function and realizes a compactification through the synchronization constraints, which is also known in the string theory. It is explained how to achieve an optimal cooperative search between the individual minimizers. The method is formulated in continuous time and related with Lagrange programming networks and cellular neural networks