Model-based face tracking for view-independent facial expression recognition

Facial expression recognition is necessary for designing any realistic human-machine interfaces. Previous published facial expression recognition systems achieve good recognition rates, but most of them perform well only when the user faces the camera and does not change his 3D head pose. We propose a new method for robust, view-independent recognition of facial expressions that does not make this assumption. The system uses a novel 3D model-based tracker to extract simultaneously and robustly the pose and shape of the face at every frame of a monocular video sequence. There are two main contributions. First, we demonstrate that the 3D information extracted through 3D tracking enables robust facial expression recognition in spite of large rotational and translational head movements (up to 90 degrees in head rotation). Second, we show that the Support Vector Machine is a suitable engine for robust classification. Recognition rates as high as 91 percent are achieved at classifying 5 distinct dynamic facial motions (neutral, opening/closing mouth, smile, raising eyebrow).