Fast training of recurrent networks based on the EM algorithm

Author

Ma, Sheng ; Ji, Chuanyi

Author_Institution

Dept. of Electr. Comput. & Syst. Eng., Rensselaer Polytech. Inst., Troy, NY, USA

Volume

9

Issue

1

fYear

1998

fDate

1/1/1998 12:00:00 AM

Firstpage

11

Lastpage

26

Abstract

In this work, a probabilistic model is established for recurrent networks. The expectation-maximization (EM) algorithm is then applied to derive a new fast training algorithm for recurrent networks through mean-field approximation. This new algorithm converts training a complicated recurrent network into training an array of individual feedforward neurons. These neurons are then trained via a linear weighted regression algorithm. The training time has been improved by five to 15 times on benchmark problems

Keywords

learning (artificial intelligence); maximum likelihood estimation; probability; recurrent neural nets; EM algorithm; expectation-maximization algorithm; learning; linear weighted regression; mean-field approximation; moving target; probabilistic model; recurrent neural networks; Adaptive systems; Approximation algorithms; Computer errors; Joining processes; Neurons; Probability density function; Process control; Systems engineering and theory; Transfer functions; Vectors;

fLanguage

English

Journal_Title

Neural Networks, IEEE Transactions on

Publisher

ieee

ISSN

1045-9227

Type

jour

DOI

10.1109/72.655025

Filename

655025