Multi-level scene understanding via hierarchical classification

In applications where the use of video surveillance is necessary and/or beneficial, it is a common goal to identify the contents of the video automatically. Of particular interest in such applications is the ability to recognize locations in the environment, where events occur, and describe the events common to those locations. This is one of the goals of scene understanding. Scene understanding is traditionally addressed from one of two separate points-of-view: the description of the underlying environment or the action taking-place throughout the scene. Each of these facets is required to address the overarching goal but, is insufficient independently to address the problem entirely. These facets are, in fact, dependent and by considering both, a more complete description becomes available. In this paper, we describe a novel, data-driven scene understanding and classification technique that captures and utilizes information about both the environment and activity within a scene.