An Efficient Global Optimization of Neural Networks by Using Hybrid Method

This paper proposes a global optimization of neural network by hybrid method. The hybrid method combines a stochastic approximation with a gradient descent method. The approximation point inclined toward a global escaping from a local minimum is estimated first by stochastic approximation, and then the update rule of Hopfield model is applied for high speed convergence as a gradient descent method. The proposed method has been applied to the 7- and 10- city traveling salesman problems, respectively. The experimental results show that the proposed method has superior convergence performances (rate and speed) to the conventional method that is Hopfield model with randomized initial neuron outputs setting. Especially, the proposed method is less affected by the initial outputs setting and so gives relatively better results than the Hopfield model as the prom size becomes larger.