Towards unsupervised detection of actions in clutter

In this paper, we describe a characterization of visual action that encodes a local photometry via a choice of interest operators and global dynamics via a realization of a stochastic dynamical model. In order to allow action detection in clutter, it is necessary for the corresponding models to have a compositional property, in that a simple action (e.g. foreground action) can be detected within a more complex one (e.g. foreground and background actions). We show that this is the case for the model we propose, which can therefore be used as a basis for building models of dynamic scenes from images without explicit supervision, by composing a complex action from a collection of elementary ones.