Unsupervised learning of micro-action exemplars using a Product Manifold

This paper presents a completely unsupervised mechanism for learning micro-actions in continuous video streams. Unlike other works, our method requires no prior knowledge of an expected number of labels (classes), requires no silhouette extraction, is tolerant to minor tracking errors and jitter, and can operate at near real time speed. We show how to construct a set of training “tracklets,” how to cluster them using a recently introduced Product Manifold distance measure, and how to perform detection using exemplars learned from the clusters. Further, we show that the system is amenable to incremental learning as anomalous activities are detected in the video stream. We demonstrate performance using the publicly-available ETHZ Livingroom data set.