Scale invariant 3D multi-person tracking using a base set of bundle adjusted visual landmarks

We present a vision system for real-time 3D tracking of multiple people moving over an extended area, as seen from a rotating and zooming camera. Despite the general problems of multiple target tracking (MTT), the use of a pan-tilt-zoom (PTZ) camera adds several difficulties for the multiplicity of connected problems. Our approach exploits multi-view image matching techniques to index and refine, at runtime, the closest world to image homography for the current view. This is made possible by applying (in a batch phase) bundle adjustment method over a set of distinctive visual landmarks extracted from the field of regard of the zooming camera sensor. The approach is experimentally evaluated on several difficult video sequences. Quantitative results show that the proposed approach makes it possible to deliver stable tracking performance in scenes of previously infeasible complexity. We achieve an almost constant standard deviation error of less than 0.3 meters in recovering 3D trajectories of multiple moving targets in an area of 70×15 meters.