Dynamics and local minima of a simple neural network for optimization

In the neural computation framework proposed by Hopfield and Tank (1985), the total energy for deriving the network dynamics is composed of multiple sub-energy functions, each of which is expressed by a quadratic form. Their work has had a great impact on various fields in science and engineering, and has produced a lot of active discussions about neural network based computation. However, problems involving local minima have not been solved yet, and so there are no methods for obtaining global optima. We can design various types of energy functions to solve an optimization problem, such as high-dimensional, low-dimensional, and/or non-linear functions. In this paper, we treat a simple total-energy function and study the nature of the minima in the dynamics. We further discuss some techniques to avoid convergence towards inadequate minima