Real-Time Activity Search of Surveillance Video

We present a fast and flexible content-based retrieval method for surveillance video. Designing a video search robust to uncertain activity duration, high variability in object shapes and scene content is challenging. We propose a two-step approach to video search. First, local motion features are inserted into an inverted index using locality-sensitive hashing (LSH). Second, we utilize a novel optimization approach based on edit distance to minimize temporal distortion, limited obscuration and imperfect queries. This approach assembles the local features stored in the index into a video segment which matches the query video. Pre-processing of archival video is performed in real-time, and retrieval speed scales as a function of the number of matches rather than video length. We demonstrate the effectiveness of the approach for counting, motion pattern recognition and abandoned object applications using a pair of challenging video datasets.