Detecting and tracking all moving objects in wide-area aerial video

Multi-megapixel cameras are transforming airborne video surveillance by enabling persistent imaging of extremely large areas while providing sufficient pixel density to resolve both vehicles and pedestrians. The sheer spatial and temporal volume of data has rendered human scanning of expansive images for miniscule moving objects intractable, underscoring the importance of automated detection and tracking systems. Existing algorithms, however, are generally designed for stationary cameras and moderately-sized objects. This paper presents one of the first systems for reliably detecting and tracking low-resolution objects of varying size and shape in challenging wide-area aerial video. Significant contributions include a simple, fast approach for robust motion detection with parallax handling; spatial-temporal filtering for quickly discarding spurious detections; adaptive shape learning for unusually-shaped objects; and multi-cue fusion for state evolution that enables tracking through confusion, occlusions, and stops. Our system is highly efficient and parallelizable, processing 1-megapixel image streams in real time on a single CPU core. Experiments on a variety of data sets demonstrate that the system outperforms more traditional detection and tracking approaches, and is able to find pedestrians missed by human ground truthers despite tiny size, poor contrast, and surrounding clutter.¹