Title :
Backpropagation through time with fixed memory size requirements
Author :
Principe, Jose C. ; Kuo, Jyh-Ming ; de Vries, Bert
Author_Institution :
Comput. NeuroEng. Lab., Florida Univ., Gainesville, FL, USA
Abstract :
A generalization of the backpropagation through time (BPTT) algorithm is presented, which, under reasonable assumptions, can lead to fixed memory size requirements. The idea is to model BPTT as the storage of activations and errors in tapped delay lines, and then generalize the tap delay line to a gamma memory. Since the depth of the gamma filter is T=K/μ, where K is the filter order and μ a scalar that can be varied between 0<μ<1, it is possible to achieve depth T with a fixed filter order K. The accuracy of the gradient computation is tested in an example
Keywords :
backpropagation; content-addressable storage; generalisation (artificial intelligence); neural nets; backpropagation through time; delay lines; errors; fixed memory size; gamma filter; gamma memory; generalization; gradient computation; learning; neural nets; Backpropagation algorithms; Biological neural networks; Boundary conditions; Convolution; Delay lines; Filters; Laboratories; Neural engineering; Real time systems; Testing;
Conference_Titel :
Neural Networks for Processing [1993] III. Proceedings of the 1993 IEEE-SP Workshop
Conference_Location :
Linthicum Heights, MD
Print_ISBN :
0-7803-0928-6
DOI :
10.1109/NNSP.1993.471868
