Efficient occlusion handling for multiple agent tracking by reasoning with surveillance event primitives

Tracking multiple agents in a monocular visual surveillance system is often challenged by the phenomenon of occlusions. Agents entering the field of view can undergo two different forms of occlusions, either caused by crowding or due to obstructions by background objects at finite distances from the camera. The agents are primarily detected as foreground blobs and are characterized by their motion history and weighted color histograms. These features are further used for localizing them in subsequent frames through motion prediction assisted mean shift tracking. A number of Boolean predicates are evaluated based on the fractional overlaps between the localized regions and foreground blobs. We construct predicates describing a comprehensive set of possible surveillance event primitives including entry/exit, partial or complete occlusions by background objects, crowding, splitting of agents and algorithm failures resulting from track loss. Instantiation of these event primitives followed by selective feature updates enables us to develop an effective scheme for tracking multiple agents in relatively unconstrained environments.