3-D Mediated Detection and Tracking in Wide Area Aerial Surveillance

We address the problem of tracking many moving objects in wide area aerial surveillance. We propose that using 3-D information significantly improves performance over standard state of the art trackers, which rely on 2-D stabilization. We present and contrast two 3-D mediated approaches. The first approach assumes a dense 3-D model has previously been obtained. Given the camera poses, we can predict the image flow between frames and perform pixel-level classification for detection. The second approach instead computes the image flow, and relies on the epipolar geometry constraint to distinguish object motion from parallax. Our experiments on real imagery show a significant improvement in probability of detection (from 36% to 77%/67%), false alarm rate (from 75% to 3%/4%), and a speedup of an order of magnitude for the tracker itself. Using explicit 3-D improves the results, at a higher computational cost.