Generalized Max Pooling for Action Recognition

Action recognition has been an important and challenging task in computer vision. Existing approaches usually employ pooling operation to encode isolated patches or trajectories and then aggregate them for a compact video presentation. In this paper, we make two contributions towards improving action recognition accuracy and efficiency. First, we study to apply a state-of-the-art pooling technique used in image classification i.e. Generalized Max Pooling (GMP) to action recognition. Second, we propose an approach to improve GMP efficiency as it is applied to videos of which the number of extracted patches is enormous. The key idea is to compute the weighted vector block-by-block by exploiting sparse encoding vectors and inverted index. Experiments on benchmark dataset, HMDB51, have shown the significant performance of GMP compared to existing pooling techniques and the efficiency improvement of our proposed approach.