Temporal modeling of facial actions from face profile image sequences

The recognition of facial action units (AUs) in image sequences is a challenging problem. AU detectors achieve good recognition rates, but virtually all of them deal only with frontal-view face images and cannot handle the temporal dynamics of AUs. We report on a system for automatic recognition of temporal models of AUs from long, profile-view, face image sequences. We exploit particle filtering to track 15 facial points in an input face-profile video sequence and we introduce facial-behavior temporal-dynamics recognition from continuous video input using temporal rules. The utilized algorithm performs both automatic segmentation and recognition of temporal segments (i.e., onset, apex, offset) of 23 AUs occurring alone or in a combination in an input face-profile video sequence. A recognition rate of 88% is achieved.