Optimal spatial sampling of hyperspectral imagery for fusion with panchromatic video in multitarget tracking

Hyperspectral imagery (HSI) data has proven useful for discriminating targets, however the relatively slow speed at which HSI data is gathered for an entire frame reduces the usefulness of fusing this information with panchromatic video. Additionally, the volume of HSI information collected affects the computational performance of software exploiting the information. Paradoxically, it is too much information and we cannot get enough of it. A new sensor under development has the potential of overcoming this problem. It has the ability to provide HSI data for a limited number of pixels while providing panchromatic video for the remainder of the pixels. The HSI data is co-registered with the panchromatic video and is available at each frame. This paper investigates the exploitation of this new sensor for target tracking. The first challenge of exploiting this sensor is to determine where the gathering of HSI data will be the most useful as compared to collecting panchromatic. We optimize the selection of pixels for which we will gather HSI data. We refer to this as spatial sampling. Spatial sampling is solved using a utility function where pixels receive a value based on their nearness to a target of interest (TOI). The TOIs are determined from the tracking algorithm providing a close coupling of the tracking and the sensor control. The weighting of the different types of TOIs is accomplished by a multiobjective genetic algorithm. Experiments compare fused versus non-fused tracking performance.