A modified hidden weight optimization algorithm for feedforward neural networks

Author

Yu, Changhua ; Manry, Michael T.

Author_Institution

Dept. of Electr. Eng., Texas Univ., Arlington, TX, USA

Volume

2

fYear

2002

fDate

3-6 Nov. 2002

Firstpage

1034

Abstract

The output weight optimization-hidden weight optimization (OWO-HWO) feedforward network training algorithm alternately solves linear equations for output weights and reduces a separate hidden layer error function with respect to hidden layer weights. Here, a new hidden layer error function is proposed which de-emphasizes net function errors that correspond to saturated activation function values. In addition, an adaptive learning rate based on the local shape of the error surface is used in hidden layer training. Faster learning convergence is experimentally verified.

Keywords

adaptive systems; convergence of numerical methods; error analysis; feedforward neural nets; learning (artificial intelligence); optimisation; OWO-HWO feedforward network training algorithm; adaptive learning rate; error surface; feedforward neural networks; hidden layer error function; hidden layer training; hidden layer weights; learning convergence; linear equations; local shape; modified hidden weight optimization algorithm; output weight optimization-hidden weight optimization; saturated activation function; Convergence; Equations; Error correction; Feedforward neural networks; Feedforward systems; Gradient methods; Joining processes; Neural networks; Shape; Training data;

fLanguage

English

Publisher

ieee

Conference_Titel

Signals, Systems and Computers, 2002. Conference Record of the Thirty-Sixth Asilomar Conference on

Conference_Location

Pacific Grove, CA, USA

ISSN

1058-6393

Print_ISBN

0-7803-7576-9

Type

conf

DOI

10.1109/ACSSC.2002.1196941

Filename

1196941