Towards a robust affect recognition: Automatic facial expression recognition in 3D faces

Facial expressions are a powerful tool that communicates a person’s emotional state and subsequently his/her intentions. Compared to 2D face images, 3D face images offer more granular cues that are not available in the 2D images. However, one major setback of 3D faces is that they impose a higher dimensionality than 2D faces. In this paper, we attempt to address this problem by proposing a fully automatic 3D facial expression recognition model that tackles the high dimensionality problem in a twofold solution. First, we transform the 3D faces into the 2D plane using conformal mapping. Second, we propose a Differential Evolution (DE) based optimization algorithm to select the optimal facial feature set and the classifier parameters simultaneously. The optimal features are selected from a pool of Speed Up Robust Features (SURF) descriptors of all the prospective facial points. The proposed model yielded an average recognition accuracy of 79% using the Bosphorus database and 79.36% using the BU-3DFE database. In addition, we exploit the facial muscular movements to enhance the probability estimation (PE) of Support Vector Machine (SVM). Joint application of feature selection with the proposed enhanced PE (EPE) yielded an average recognition accuracy of 84% using the Bosphorus database and 85.81% using the BU-3DFE database, which is statistically significantly better (at p < 0.01 and p < 0.001 , respectively) if compared to the individual exploit of the optimal features only.