Dynamic Bayesian activity modeling in video via multi-feature integration

We present a Bayesian approach to unsupervised clustering of activity within video imagery. Vehicles and pedestrians are tracked within the video imagery and their collective activity in each time frame is measured and categorized using a natural extension of the dynamic latent Dirichlet allocation model. Our extension involves use of multiple types of simultaneously observed features from multiple classes of objects within the video imagery. Within the prior for the model these features are treated as independent, and modeled as draws from a variety of appropriate distribution types. By including multiple features, the model generates a richer set of activities; we quantitatively show that this yields better predictions of physical attributes within the scene, relative to currently available models that use only the single best feature. We show this by comparing model prediction of traffic light states within a busy intersection, which we ground-truth manually within the video imagery.