Who is doing what? Simultaneous recognition of actions and actors

Recognizing human actions in videos has become a rapidly growing area of research. Most existing research has focused only on a single aspect i.e. recognition of actions. However, humans tend to perform different actions in their own styles. In this paper, we deal with the problem of simultaneously identifying actions and the underlying styles (actors) in videos. We propose a hierarchical approach based on conventional action recognition and asymmetric bilinear modeling. Our approach is solely based on dynamics of the underlying activity. Results on the multi-actor multi-action data set IX-MAS show a high recognition rate.