Title :
Chaotic Time Series Prediction Based on a Novel Robust Echo State Network
Author :
Decai Li ; Min Han ; Jun Wang
Author_Institution :
Fac. of Electron. Inf. & Electr. Eng., Dalian Univ. of Technol., Dalian, China
fDate :
5/1/2012 12:00:00 AM
Abstract :
In this paper, a robust recurrent neural network is presented in a Bayesian framework based on echo state mechanisms. Since the new model is capable of handling outliers in the training data set, it is termed as a robust echo state network (RESN). The RESN inherits the basic idea of ESN learning in a Bayesian framework, but replaces the commonly used Gaussian distribution with a Laplace one, which is more robust to outliers, as the likelihood function of the model output. Moreover, the training of the RESN is facilitated by employing a bound optimization algorithm, based on which, a proper surrogate function is derived and the Laplace likelihood function is approximated by a Gaussian one, while remaining robust to outliers. It leads to an efficient method for estimating model parameters, which can be solved by using a Bayesian evidence procedure in a fully autonomous way. Experimental results show that the proposed method is robust in the presence of outliers and is superior to existing methods.
Keywords :
belief networks; chaos; learning (artificial intelligence); mathematics computing; optimisation; recurrent neural nets; time series; Bayesian evidence procedure; Bayesian framework; ESN learning; Gaussian likelihood function; Laplace distribution; Laplace likelihood function; RESN training; bound optimization algorithm; chaotic time series prediction; echo state mechanisms; model output likelihood function; model parameter estimation; outliers; robust echo state network; robust recurrent neural network; surrogate function; Bayesian methods; Mathematical model; Optimization; Reservoirs; Robustness; Training; Training data; Echo state network (ESN); Laplace likelihood function; robust model; surrogate function;
Journal_Title :
Neural Networks and Learning Systems, IEEE Transactions on
DOI :
10.1109/TNNLS.2012.2188414