Title :
A fast hybrid algorithm of global optimization for feedforward neural networks
Author :
Minghu, Jiang ; Xiaoyan, Zhu ; Baozong, Yuan ; Xiaofang, Tang ; Biqin, Lin ; Qiuqi, Ruan ; Mingyan, Jiang
Author_Institution :
Inst. of Inf. Sci., Northern Jiaotong Univ., Beijing, China
Abstract :
This paper presents the hybrid algorithm of global optimization of dynamic learning rate for multilayer feedforward neural networks (MLFNN). The effect of inexact line search on conjugacy was studied and a generalized conjugate gradient method based on this effect was proposed and shown to have global convergence for error backpropagation of MLFNN. The descent property and global convergence was given for the improved hybrid algorithm of the conjugate gradient algorithm, the results of the proposed algorithm show a considerable improvement over the Fletcher-Reeves algorithm and the conventional backpropagation (BP) algorithm, it overcomes the drawback of conventional BP and Polak-Ribieve conjugate gradient algorithm that maybe plunge into local minima
Keywords :
backpropagation; conjugate gradient methods; convergence of numerical methods; feedforward neural nets; optimisation; BP algorithm; Fletcher-Reeves algorithm; Polak-Ribieve conjugate gradient algorithm; backpropagation algorithm; conjugate gradient algorithm; descent property; dynamic learning rate; error backpropagation; fast hybrid algorithm; feedforward neural networks; generalized conjugate gradient method; global convergence; global optimization; inexact line search; multilayer feedforward neural networks; Acceleration; Backpropagation algorithms; Computer networks; Convergence; Feedforward neural networks; Gradient methods; Hybrid intelligent systems; Intelligent networks; Multi-layer neural network; Neural networks;
Conference_Titel :
Signal Processing Proceedings, 2000. WCCC-ICSP 2000. 5th International Conference on
Conference_Location :
Beijing
Print_ISBN :
0-7803-5747-7
DOI :
10.1109/ICOSP.2000.893409
