On Runge-Kutta neural networks: Training in series-parallel and parallel configuration

Recently the Runge-Kutta neural network (RKNN) in series-parallel configuration for identification of ordinary differential equation (ODE) was introduced. The neural network is constructed according to the Runge-Kutta approximation method whereby a precise estimate of the changing rates of the system states is possible. In this contribution we extend the approach of to a general state space representation with output equation and develop algorithms to calculate the Jacobian and the gradient for the dynamic neural network in series-parallel and parallel configuration. Using parallel configuration instead of series-parallel configuration the identification of systems with (partially) unknown states becomes possible, which is an important aspect in real-life system identification tasks. The benefits, in terms of estimation accuracy and robustness to noise, of networks trained in parallel configuration over networks trained in series-parallel configuration (as proposed in) are shown by simulations.