Facial Expression Analysis and Synthesis : a Bilinear Approach

This paper introduces a bilinear model to analyze and transfer expression or identity of 3D faces, and its applications in 3D and 2D areas. Our aim is to separate identity and expression factors of face data into two independent linear subspaces. First all the data are proceeded to have vertex-to-vertex correspondences. We build a morphable face model to transform these dense-vertex data into a low-dimension parameter representation. Based on these parameter vectors denoting special faces, we train a bilinear identity-expression model, which spans training expressions and identities by vectors of only a few dimensions. In 3D applications, we need to estimate the optimal parameters for unknown factor of new shapes first, then change corresponding factor to transact various tasks, such as expression synthesis, expression extraction, expression clone, et al. Furthermore, we recover the 3D shape of a face in a frontal photograph, by a 2D-3D feature-based fitting procedure. Then by changing the expression of 3D shape, we synthesize new images with different expression states of input face, which are used for expression synthesis or identity recognition. The results of experiment show that our work is quite enlightening in applications involved with facial expressions.