Sports Video Mining via Multichannel Segmental Hidden Markov Models

We study sports video mining as a machine learning and statistical inference problem. We focus on mid-level semantic structures that can serve as building blocks for high-level semantic analysis. Particularly, we are interested in how to infer multiple coexistent structures jointly. We present a new multichannel segmental hidden Markov model (MCSHMM) that is a unique probabilistic graphical model with two advantages. One is the integration of both hierarchical and parallel dynamic structures that offers more flexibility and capacity of capturing the interaction between multiple Markov chains. The other is the incorporation of the segmental HMM (SHMM) to deal with variable-length observations. In addition, we develop a maximum a posteriori (MAP) estimator to optimize the model structure and parameters simultaneously. The proposed MCSHMM is used for American football video analysis. The experiment result shows that the MCSHMM outperforms existing HMMs and has potential to be extended for other video mining tasks.