Combining Localized Oriented Rectangles and Motion History Image for Human Action Recognition

We present a method for action recognition that combines and thus shares the advantages of both local and global representation of the video sequence. The dense Harris corners are first extracted as the local interest points, which are then masked by the motion history image (MHI). Next, a set of rectangular filters with different orientations and rotation points are applied on these masked local points, and the orientation of the rectangle with highest response at the interest points is recorded. At last, a grid based global descriptor is proposed to estimate the distribution of the rectangle patches, which is referred to as histogram of localized oriented rectangles (HLOR). In order to cover enough temporal information, the histogram of oriented gradient (HOG) of the MHI is also calculated with the same grid partition as HLOR. Final representation of the video is a concatenation of HLOR and MHI-HOG. The performance of our method is evaluated on Weizmann and KTH datasets. The state-of-art recognition results are obtained.