Object-event graph matching for complex activity recognition

In security, the most relevant criminal and terrorist activities are often of high complexity: they involve several entities interacting sequentially and simultaneously over an extended time interval. In this paper, we present a powerful approach for complex activity recognition and analysis using graph representation and matching. It is based on the representation of activities in terms of objects, events and processes, which are modeled as nodes of an attributed relational graph (ARG). The recognition of complex activities, taking into account observation uncertainty and incompleteness, is performed using graph matching of template graphs and the data graph. The data graph represents observations of objects, events and processes collected from low-level signal processing and other information sources. The models of the complex activities to be detected are represented as template graphs. Markov chain Monte Carlo sampling is proposed to infer probabilities of activity occurrence, object involvement and event occurrence for detection, event prediction and sensor management in complex activity recognition problems. The suggested method is illustrated using a toy example from maritime surveillance.