On stable learning of block-diagonal recurrent neural networks, Part 1: the RENNCOM algorithm

A novel learning algorithm, the RENNCOM (recurrent neural network constrained optimization method), is suggested in this paper, for training block-diagonal recurrent neural networks. The training task is formulated as a constrained optimization problem, whose objective is twofold: (i) minimization of an error measure, leading to successful approximation of the input/output mapping and (ii) optimization of an additional functional, which aims at ensuring network stability throughout the learning process. The characteristics of the proposed algorithm are highlighted by a simulation example, where a nonlinear dynamic identification problem is presented.