Multiple sequence behavior recognition on humanoid robot using long short-term memory (LSTM)

Recurrent neural networks (RNN) are powerful sequence learners. However, RNN suffers from the problem of vanishing gradient point. This fact makes learning sequential task more than 10 time steps harder for RNN. Recurrent network with LSTM cells as hidden layers (LSTM-RNN) is a deep learning recurrent network architecture designed to address the vanishing gradient problem by incorporating memory cells (LSTM cells) in the hidden layer(s). This advantage puts it at one of the best sequence learners for time-series data such as cursive hand writings, protein structure prediction, speech recognition and many more task that require learning through long time lags [2][3][4], In this paper, we applied the concept of using recurrent networks with LSTM cells as hidden layer to learn the behaviours of a humanoid robot based on multiple sequences of joint data from 10 joints on the NAO robot. We show that the LSTM network is able to learn the patterns in the data and effectively classify the sequences into 6 different trained behaviors.