Spatial-temporal correlation for trajectory based action video retrieval

The bag-of-visual-words model has been widely utilized for content based image and video retrieval due to its scalability. In this paper, we extend this model for human action video retrieval. We adopt dense trajectory features which are able to achieve the state-of-the-art performance on action recognition, while most of the existing video retrieval methods utilize descriptors of local interest points. In order to improve similarity measurement between bag-of-visual-words model based representation, we propose to discover and incorporate spatial-temporal correlation (STC) among the trajectories in a given query video. The spatial-temporal correlation consists of spatial proximity and temporal consistence among trajectories, which is capable of strengthening discriminative power among visual words. Note that such query focused spatial-temporal correlation makes our method dynamic for different queries and is able to improve retrieval performance without significantly increasing the size of a visual vocabulary. The experimental results on an action video dataset demonstrate that our proposed method outperforms other similar methods.