GPU-accelerated hierarchical dense correspondence for real-time aerial video processing

Video from aerial surveillance can provide a rich source of data for many applications and can be enhanced for display and analysis through such methods as mosaic construction, super-resolution, and mover detection. All of these methods require accurate frame-to-frame registration, which for live use must be performed in real time. In many situations, scene parallax may make alignment using global transformations impossible or error-prone, limiting the performance of subsequent processing and applications. For these cases, dense (per-pixel) correspondence is required, but this can be computationally prohibitive. This paper presents a hierarchical dense correspondence algorithm designed for implementation on graphics processing units (GPUs). Since the method does not rely on epipolar geometry, it is also suitable for use when there are uncor-rected nonlinear lens distortions. A framework for using this dense correspondence to implement local mosaicking, super-resolution enhancement, and mover detection is also presented and demonstrated using examples of each of these types of enhancement and different types of video sources.