A PDM based approach to recovering 3D face pose and structure from video

We present a face pose and structure estimation algorithm based on using a point distribution model (PDM) for 3D face features. The 3D PDM is constructed from a set of laser-scanned 3D head models. By using the PDM for the facial feature points, a novel algorithm was developed to recover the 3D face structure and pose from a single image. Once the 3D face structure was obtained, the facial feature points tracked from video are used to determine the temporal face poses. For the proposed face pose and structure estimation, the rotation matrix and translation vector represent the pose information and the coefficients for the PDM control the face structure through a linear combinational form. For a given rotation matrix, the PDM coefficients and the translation vector can be uniquely determined from the tracked feature points, therefore we search for the best combination of rotation angles in an outer loop of the optimization procedure. After the 3D face structure is computed, the 3D face pose tracking is accomplished through a dynamic pose estimation process. Experimental results of applying the proposed algorithms to both simulated and real data are shown to demonstrate its performance.