Functional scene element recognition for video scene analysis

We present a method to detect and recognize functional scene elements in video scenes. A functional scene element is a location or object that is primarily defined by its specific function or purpose, rather than its appearance or shape. Our method combines techniques from video scene analysis with functional recognition to decompose a video scene into its functional elements such as parking spots, building entrances, roads and sidewalks. Existing techniques for functional object recognition in video are designed for high-resolution video with little clutter and constrained situations, while our approach is designed for real-world video surveillance scenes where there are many movers, and detection and tracking can be poor because of low resolution and frame rates. Video scene analysis methods are focused on motion pattern learning and anomaly detection, whereas we take a recognition approach and develop motion pattern models for specific functional categories. The movements of objects such as vehicles and pedestrians are exploited to detect and classify functional scene elements in an online process that probabilistically accumulates evidence over many tracks to compensate for noisy and partial observations. Results are shown on simulated and real data of complex, busy scenes containing multiple instances of different functional objects. The detected elements are then used to demonstrate that building activity profiles can be extracted and used to distinguish different types of buildings.