HO2: A new feature for multi-agent event detection and recognition

In this paper, we present a new feature to model a class of events that consist of complex interactions among multiple entities captured by tracks and inter-object relationships over space and time. Existing approaches represent these events using features that measure only pairwise relationships between entities at a time, such as relative distance and relative speed. Due to the limitations of the pairwise entity relationship descriptors, this class of events is mainly defined and recognized using rule-based approach. The new feature, Histogram of Oriented Occurrences (HO2), captures the interactions of all entities of interests in terms of configurations over space and time. HO2 features encapsulate entity tracks, inter-object relationships and the context of the environment into a spatial distribution that characterizes the corresponding event. HO2 feature is a compact and structured descriptor for capturing multi-object relationships. We demonstrate its value in complex event detection and recognition using standard statistical clustering and classification techniques.