Illumination aware MCMC Particle Filter for long-term outdoor multi-object simultaneous tracking and classification

This paper addresses real-time automatic visual tracking, labeling and classification of a variable number of objects such as pedestrians or/and vehicles, under time-varying illumination conditions. The illumination and multi-object configuration are jointly tracked through a Markov Chain Monte-Carlo Particle Filter (MCMC PF). The measurement is provided by a static camera, associated to a basic foreground / background segmentation. As a first contribution, we propose in this paper to jointly track the light source within the Particle Filter, considering it as an additionnal object. Illumination-dependant shadows cast by objects are modeled and treated as foreground, thus avoiding the difficult task of shadow segmentation. As a second contribution, we estimate object category as a random variable also tracked within the Particle Filter, thus unifying object tracking and classification into a single process. Real time tracking results are shown and discussed on sequences involving various categories of users such as pedestrians, cars, light trucks and heavy trucks.