Early Facial Expression Recognition Using Hidden Markov Models

Although it is often necessary to recognize users´ expressions as soon as possible after it starts and before it ends in many applications, few methods have been proposed explicitly for early facial expression recognition. In this paper, we propose an early facial expression recognition method by using Hidden Markov Model. The relative displacement of the feature points between the current frame and the neutral frame are extracted as the facial features. During training, an iterative algorithm is introduced to find a classification entropy threshold and model parameters of early HMM. During testing, an image sequence is assigned an expression category when the entropy of the expression likelihood obtained from early HMMs is below the threshold by gradually increasing sequence length. Experimental results on CK+ and MMI databases show the effectiveness of our approach.