Prototype-Based Modeling for Facial Expression Analysis

Automatic facial expression analysis systems are aiming towards the application of computer vision techniques in human computer interaction, emotion analysis, and even medical care via a space mapping between the continuous emotion and a set of discrete expression categories. The main difficulty with these systems is the inherent problem of facial alignment due to person-specific appearance. Beside the facial representation problem, the same displayed facial expression may vary differently across humans; this can be true even for the same person in different contexts. To cope with these variable factors, we introduce the concept of prototype-based model as anchor modeling through a SIFT-flow registration. A set of prototype facial expression models is generated as a reference space of emotions on which face images are projected to generate a set of registered faces. To characterize the facial expression appearance, oriented gradients are processed on each registered image. We obtained the best results 87% with the person-independent evaluation strategy on JAFFE dataset (7-class expression recognition problem), and 83% on the complex setting of the GEMEP-FERA database (5-class problem).