Appearance-based action recognition in the tensor framework

There are multiple contributory factors taking place in an action video, e.g., person, clothing, illumination, etc. When these factors change together, conventional 1-mode analysis like PCA in action space encounters difficulties. The N-mode analysis overcomes this problem. In this paper, we propose a novel framework for recognition of actions using silhouettes based on N-mode SVD. We use the silhouette ensembles to form a 3^rd order tensor comprising three modes: pixels, actions and people. Using N-mode SVD, we find the bases as well as the coefficients for the action space. For a query sequence, the resulting action-mode coefficients are compared with the learned coefficients to find the action class. Through experiments on a common database, we compare the proposed method with 1-mode PCA in appearance-base recognition of human actions and show that our method outperforms 1-mode analysis.