A fast hybrid algorithm of global optimization for feedforward neural networks

This paper presents the hybrid algorithm of global optimization of dynamic learning rate for multilayer feedforward neural networks (MLFNN). The effect of inexact line search on conjugacy was studied and a generalized conjugate gradient method based on this effect was proposed and shown to have global convergence for error backpropagation of MLFNN. The descent property and global convergence was given for the improved hybrid algorithm of the conjugate gradient algorithm, the results of the proposed algorithm show a considerable improvement over the Fletcher-Reeves algorithm and the conventional backpropagation (BP) algorithm, it overcomes the drawback of conventional BP and Polak-Ribieve conjugate gradient algorithm that maybe plunge into local minima