An EM-based algorithm for recurrent neural networks

Author

Ma, Sheng ; Ji, Chuanyi

Author_Institution

Dept. of Electr. Comput. & Syst. Eng., Rensselaer Polytech. Inst., Troy, NY, USA

fYear

1995

fDate

17-22 Sep 1995

Firstpage

175

Abstract

A stochastic model is established for fully-connected recurrent neural networks with sigmoid units based on Gibbs distributions. The EM (expectation-maximization) algorithm with a mean field approximation is then applied to train recurrent networks through hidden state estimation. The resulting EM-based algorithm, which reduces training the original recurrent network to training a set of individual feedforward neurons, simplifies the original training process and reduces the training time

Keywords

approximation theory; feedforward neural nets; learning (artificial intelligence); recurrent neural nets; state estimation; stochastic processes; EM-based algorithm; Gibbs distributions; backpropagation; expectation-maximization algorithm; feedforward neurons; fully-connected recurrent neural networks; hidden state estimation; mean field approximation; recurrent networks training; recurrent neural networks; sigmoid units; stochastic model; training time reduction; Computer networks; Convergence; Information theory; Jacobian matrices; Neural networks; Neurons; Recurrent neural networks; Testing;

fLanguage

English

Publisher

ieee

Conference_Titel

Information Theory, 1995. Proceedings., 1995 IEEE International Symposium on

Conference_Location

Whistler, BC

Print_ISBN

0-7803-2453-6

Type

conf

DOI

10.1109/ISIT.1995.531524

Filename

531524