Stabilizing and Improving the Learning Speed of 2-Layered LSTM Network

This paper presents a novel method to initialize the LSTM network weights in order to improve and stabilize the learning speed, based on Nguyen and Widrowpsilas work for MLP networks. The derived equations for weight initialization are based on the study of the behavior of the memory cells output in the hidden layer. To test and evaluate the proposed method, we use a 2-Layered LSTM network to approximate one and two dimensional real non-linear functions. The obtained results show that our initialization method improves the training process.