A New Hybrid Synthetic Aperture Imaging Model for Tracking and Seeing People Through Occlusion

Robust detection and tracking of multiple people in cluttered and crowded scenes with severe occlusion is a significant challenge for many computer vision applications. In this paper, we present a novel hybrid synthetic aperture imaging model to solve this problem. The main characteristics of this approach are as follows. 1) To the best of our knowledge, this is the first attempt to solve the occluded people imaging and tracking problem in a joint multiple camera synthetic aperture imaging domain. 2) A multiple model framework is designed to achieve seamless interaction among the detection, imaging and tracking modules. 3) In the object detection module, a multiple constraints-based approach is presented for people localization and ghost objects removal in a 3-D foreground silhouette synthetic aperture imaging volume. 4) In the synthetic imaging module, a novel occluder removal-based synthetic imaging approach is proposed to significantly improve the imaging quality of objects even under severe occlusion. 5) In the object tracking module, a camera array is used for robust people tracking in color synthetic aperture images. A network-camera-based hybrid synthetic aperture imaging system has been set up, and experimental results with qualitative and quantitative analyses demonstrate that the method can reliably locate and see people in challenging scenes.