Segmentation and tracking of multiple humans in complex situations

Segmenting and tracking multiple humans is a challenging problem in complex situations in which extended occlusion, shadow and/or reflection exists. We tackle this problem with a 3D model-based approach. Our method includes two stages, segmentation (detection) and tracking. Human hypotheses are generated by shape analysis of the foreground blobs using a human shape model. The segmented human hypotheses are tracked with a Kalman filter with explicit handling of occlusion. Hypotheses are verified while being tracked for the first second or so. The verification is done by walking recognition using an articulated human walking model. We propose a new method to recognize walking using a motion template and temporal integration. Experiments show that our approach works robustly in very challenging sequences.