Real-time integrated multi-object detection and tracking in video sequences using detection and mean shift based particle filters

Object detection and tracking have been studied separately in most cases. This paper presents a new method integrating generic object detection with particle filtering based tracking algorithm in one consistent framework to achieve real time robust multi-object tracking (MOT) in video sequences. By using detection, we can not only do initialization automatically and dynamically, but also solve the data association problem for MOT easily. To improve the degeneracy problem which most particle filtering methods suffer with, we incorporate the strength of resampling, proposed detection based optimal importance function, and mean shift mode seeking together to make particles much more efficient and estimate the posterior density better. The detection result gives the global optimal of the posterior density while the mean shift mode seeking finds the local optimal. Experimental results show the superior performance of our approach to the available tracking methods.