View-invariant human activity recognition based on shape and motion features

Recognizing human activities from image sequences is an active area of research in computer vision. Most of the previous work on activity recognition focuses on recognition from a single view and ignores the issue of view invariance. In this paper, we present a view invariant human activity recognition approach that uses both motion and shape information for activity representation. For each frame in the video, a 128 dimensional optical flow vector of the region of interest is used to represent the motion of the human body, and a 90 dimensional eigen-shape vector is used to represent the shape. Each activity is represented by a set of hidden Markov models (HMMs), where each model represents the activity from a different viewing direction, to realize view-invariance recognition. Experiments on a database of video clips of different activities show that our method is robust.