Comparative study on EKF training algorithm with EM and MLE for ANN modeling of nonlinear systems

The system identification/modeling problem looks for a suitably parameterized model, representing a given process. The parameters of the model are adjusted to optimize a performance function based on error between the given process output and identified process output. The linear system identification field is well established with many classical approaches whereas most of those methods cannot be applied for nonlinear systems. The problem becomes tougher if the system is completely unknown with only the output time series is available. It has been reported that the capability of Artificial Neural Network to approximate all linear and nonlinear input-output maps makes it predominantly suitable for the blind identification of nonlinear systems, where only the output time series is available. The work reported here is an attempt for modeling certain nonlinear systems using recurrent neural networks, in which the network parameters are estimated using the Extended Kalman Filtering (EKF) and an extension of the same with EKF with Expectation Maximization (EM) (to alleviate the problems encountered in Kalman filtering). The paper also compares the performance of the neural network model implemented with Maximum Likelihood Estimation (MLE), An assessment on the model performances in terms of the mean square error (MSE) and computational complexity has also been done for these algorithms.