Efficient Hidden Semi-Markov Model Inference for Structured Video Sequences

The semantic interpretation of video sequences by computer is often formulated as probabilistically relating lower-level features to higher-level states, constrained by a transition graph. Using hidden Markov models inference is efficient but time-in-state data cannot be included, whereas using hidden semi-Markov models we can model duration but have inefficient inference. We present a new efficient O(T) algorithm for inference in certain HSMMs and show experimental results on video sequence interpretation in television footage to demonstrate that explicitly modelling time-in-state improves interpretation performance