From video to text: Semantic driving scene understanding using a coarse-to-fine method

Semantic understanding from video is one of the most challenging tasks in video analysis. However, it has not been taken enough attention. In this paper, we focus on understanding the semantics of video in the driving scene. We present a coarse-to-fine method to parse the driving scene, and obtain the high-level semantic information of the scene. In the coarse phase, we divide the captured frame into four separate parts based on edge density entropy and scene context. In the fine phase, we join multi-class object segmentation and detection algorithms together in a unified Conditional Random Filed (CRF) model for each part understanding. Moreover, the object probabilistic location prior knowledge based on training and previous edge density entropy result is also integrated into our approach for better object localization. Experimental results show that our proposed method is effective comparing to current state-of-the-art approaches.