Task oriented facial behavior recognition with selective sensing

Facial behaviors represent activities of face or facial feature in spatial or temporal space, such as facial expressions, face pose, gaze, and furrow happenings. An automated system for facial behavior recognition is always desirable. However, it is a challenging task due to the richness and ambiguity in daily facial behaviors. This paper presents an efficient approach to real-world facial behavior recognition. With dynamic Bayesian network (DBN) technology and a general-purpose facial behavior description language (e.g., FACS), a task oriented framework is constructed to systematically represent facial behaviors of interest and the associated visual observations. Based on the task oriented DBN, we can integrate analysis results from previous times and prior knowledge of the application domain both spatially and temporally. With the top–down inference, the system can make dynamic and active selection among multiple visual channels. With the bottom–up inference from observed evidences, the current facial behavior can be classified with a desired confidence via belief propagation. We demonstrate the proposed task-oriented framework for monitoring driver vigilance. Experimental results demonstrate the validity and efficiency of our approach.