Intensity score for facial actions detection in near-frontal-view face sequences

This paper proposes a method to detect the facial Action Units (AUs) and introduce an automatic measurement in predicting the intensity scores of each AU in near-to-frontal face image sequences. To our knowledge, this is the first attempt in computer vision to automate the intensity scores of facial expression research. First, the facial feature points are detected by using Gabor feature based boosted classifiers and the movement of each point is tracked by optical flow. Then, we introduce a set of distance measurement for the feature points and analyze the distances by using Sequence Analysis. Further, we exploit the sequence partition to predict the possible temporal segments in the sequence. We found there is a relationship between the intensity scores and the partition threshold, which we automated the process of threshold selection in our work. We tested the proposed prototype with our in-house dataset and MMI database. Finally, we discuss the result, the possibilities of further research, and the next challenges for computer vision scientist in facial actions detection.