Semantic video event search for surveillance video

We present a distributed framework of understanding, indexing, and searching complex events from large amounts of surveillance video content. Video events and relationships between scene entities are represented by Spatio-Temporal And-Or Graphs (ST-AOG) and inferred in a distributed computing system using a bottom-up top-down strategy. We propose a method for sub-graph indexing of ST-AOGs of the recognized events for robust retrieval and quick search. Plain text reports of the scene are automatically generated to describe scene entities´ relationships, contextual information, as well as events of interest. When a query is provided as keywords, plain text, voice, or a video clip, the query is parsed and the closest events are extracted utilizing text description and sub-graph matching.