Recognizing Actions with Multi-view 2D Observations Recovered from Depth Maps

Depth maps based action recognition has been received much research attention in recent years due to its robustness to environmental elements in capturing and its relatively well performance in protecting user´s privacy. Taking the captured sequential depth maps as inputs, we propose a framework in this paper to recognize actions from such data. We first recover multi-view 2D observations in each frame of the sequence and then accumulate them by employing motion energy image (MEI) in each observing view. Action features that combines occupancy and motion descriptors are extracted for capturing the discriminative patterns in resulted MEIs, and then fed to action modeling based on Gaussian Mixture Model (GMM) or recognition based on Bayesian theory. Experimental results on MSR Action3D dataset show a better recognition performance by the proposed framework, compared with three competitors, which reveals its effectiveness and priority in depth maps based action recognition.