View-robust action recognition based on temporal self-similarities and dynamic time warping

In this paper, we propose an approach for human action recognition based on self-similarities of actions and dynamic-time warping method. To recognize actions under arbitrary views, we use a recent self-similarity matrix (SSM) method. Through analyzing the essence of SSMs we find that the SSMs capture a wealth of global time information useful for action recognition robust to viewpoints. The dynamic-time warping (DTW) algorithm is applied to make full use of the time information contained in SSMs. After performing DTW, we compute a collection of distances corresponding to mapped set of descriptors between the test sequence and all training sequences. Then the k-nearest neighbor classifier (KNNC) is implemented to classify the test action. We validated our method on the public multi-view IXMAS dataset and obtained promising results compared to the state-of-the-art bag-of-feature-based method.