δ-NARMA neural networks: a new approach to signal prediction

This paper presents a new connectionist architecture for stochastic univariate signal prediction. After a review of related statistical and connectionist models pointing out their advantages and limitations, we introduce the ε-NARMA model as the simplest nonlinear extension of ARMA models. These models then provide the units of a MLP-like neural network: the δ-NARMA neural network. The associated learning algorithm is based on an extension of classical backpropagation and on the concept of virtual error. Such networks can be seen as an extension of ARIMA and ARARMA models and face the problem of nonstationary signal prediction. A theoretical study brings understanding of experimental phenomena observed during the δ-NARMA learning process. The experiments carried out on three railroad-related real-life signals suggest that δ-NARMA networks outperform other studied univariate models