Training feedforward neural networks with the Dogleg method and BFGS Hessian updates

Author

Perantonis, S.J. ; Ampazis, N. ; Spirou, S.

Author_Institution

Inst. of Inf. & Telecommun., Nat. Center for Sci. Res. DEMOKRITOS, Athens, Greece

Volume

1

fYear

2000

fDate

2000

Firstpage

138

Abstract

We introduce an advanced optimization algorithm for training feedforward neural networks. The algorithm combines the Broyden-Fletcher-Goldfarb-Shanno (BFGS) Hessian update formula with a special case of trust region techniques, called the Dogleg method, as an alternative technique to line search methods. Simulations regarding classification and function approximation problems are presented which reveal a clear improvement both in convergence and success rates over standard BFGS implementations

Keywords

Hessian matrices; convergence; feedforward neural nets; function approximation; learning (artificial intelligence); optimisation; pattern classification; BFGS Hessian updates; Broyden-Fletcher-Goldfarb-Shanno Hessian update formula; Dogleg method; advanced optimization algorithm; classification; success rates; trust region techniques; Convergence; Cost function; Electronic mail; Feedforward neural networks; Function approximation; Globalization; Informatics; Mathematics; Neural networks; Search methods;

fLanguage

English

Publisher

ieee

Conference_Titel

Neural Networks, 2000. IJCNN 2000, Proceedings of the IEEE-INNS-ENNS International Joint Conference on

Conference_Location

Como

ISSN

1098-7576

Print_ISBN

0-7695-0619-4

Type

conf

DOI

10.1109/IJCNN.2000.857827

Filename

857827