Self-similarity based action recognition using Conditional Random Fields

An extensive amount of research is being undertaken to gracefully solve the Human action recognition problem. To this end, in this paper, we introduce the application of self- similarity surfaces for human action recognition. These surfaces were introduced by Shechtman & Irani (CVPR´07) in the context of matching similarities between images or videos. These surfaces are obtained by matching a small patch, centered at a pixel, to its larger surroundings, aim- ing to capture similarities of a patch to its neighborhood. Once these surfaces are computed, we propose to transform these surfaces into Histograms of Oriented Gradients (HoG), which are then used to train Conditional Random Fields (CRFs). Our novelty lies in recognizing the utility of these selfsimilarity surfaces for human action recognition. In addition, in contrast to Shechtman & Irani (CVPR´07), we compute only a few of these surfaces (two per frame) for our task. The proposed method does not rely on the structure recovery nor on the correspondence estimation, but makes only mild assumptions about the rough localization of a per- son in the frame. We demonstrate good results on a publicly available dataset and show that our results are comparable to other well known works in this area.