Semantic Event Retrieval from Surveillance Video Databases

This paper proposes a framework for retrieving semantic video events from indoor surveillance video databases. The goal is to locate video sequences containing events of interest to the user. This framework starts by tracking objects and segmenting videos into Common Appearance Intervals (CAIs). The spatiotemporal trajectories are obtained, based on which features are extracted for the construction of semantic event models. In the retrieval, the database user interacts with the machine and provides "feedbacks" to the retrieval result. The learning component learns from the spatiotemporal data, the semantic event model as well as the "feedback" and returns the refined result to the user. Specifically, the learning algorithm is developed based on a Coupled Hidden Markov Model (CHMM), which models the interactions of objects in CAIs and recognizes hidden patterns among them. This iterative learning and retrieval process contributes to the bridging of the "semantic gap", and the experimental results show the effectiveness of the proposed framework.