Semantic Event Detection using Conditional Random Fields

Semantic event detection is an active research field of video mining in recent years. One of the challenging problems is how to effectively model temporal and multi-modality characteristics of video. In this paper, we employ Conditional Random Fields (CRFs) to fuse temporal multi-modality cues for event detection. CRFs are undirected probabilistic models designed for segmenting and labeling sequence data. Compared with traditional SVM and Hidden Markov Models (HMMs), CRFs based event detection offers several particular advantages including the abilities to relax strong independence assumptions in the state transition and avoid a fundamental limitation of directed graphical models. To detect event, we use a three-level framework based on multi-modality fusion and mid-level keywords. The first level extracts audiovisual features, the mid-level detects semantic keywords, and the high-level infers semantic events from multiple keyword sequences. The experimental results from soccer highlights detection demonstrate that CRFs achieves better performance particularly in slice level measure.