Multi-spectral detection and tracking in cluttered urban environments

Automatic detection and tracking of moving targets in full motion video (FMV) from aerial imaging systems has significant interest in the defense and security community. However often times performance is degraded in a given spectral band due to environmental conditions and poor target response in a given band. The overall goal of this work is to increase the probability of detection and track association in cluttered urban environments while simultaneously suppressing false alarms by fusing the detection results and features from different spectral bands. We use a Gaussian mixture model (GMM) to detect background pixels, and define potential targets as being in regions that are found to be non-background. Detections from each spectral band are fused to form multi-spectral target candidates. Detected target candidates are associated with targets from a tracking database by matching features from the scale-invariant feature transform (SIFT). We create tracking profiles consisting of location history and vector velocity history for all targets in the scene. This algorithm was evaluated with synthetically generated datasets from the Digital Imaging and Remote Sensing Image Generation (DIRSIG) software model producing visible, near infrared, mid-wave infrared and long-wave infrared FMV that include moving vehicles in an urban environment. The proposed fusion algorithm provides a detection rate over 82%, while decreasing incorrect associations in cluttered areas such as intersections or partial occlusions where a portion of the vehicle is hidden from sensor view. This paper will describe the approach and demonstrate the performance with simulated DIRSIG FMV data.