A global convergence PSO training algorithm of neural networks

Traditional gradient-based training algorithms have been known to suffer from local minima and have heavy computation load for obtaining the derivative information. The particle swarm optimization (PSO) method was used as a training algorithm of neural networks to improve the convergence rate. However, as the network architecture grows, the size of swarm increases exponentially, which increase the computational complexity evidently. Moreover, such algorithms had the problem of premature convergence. An improved PSO training algorithm was proposed in this paper. The swarm was only composed of two particles in the new algorithm. The algorithm was guaranteed to converge to the global optimization solution with probability one. Simulation results show the new algorithm has fast convergence rate and high accuracy. Moreover, the convergence of the algorithm didn´t depend on the initial value of weights of neural networks.