Extracting 3D facial animation parameters from multiview video clips

We propose an accurate and inexpensive procedure that estimates 3D facial motion parameters from mirror-reflected multiview video clips. We place two planar mirrors near a subject´s cheeks and use a single camera to simultaneously capture a marker´s front and side view images. We also propose a novel closed-form linear algorithm to reconstruct 3D positions from real versus mirrored point correspondences in an uncalibrated environment. Our computer simulations reveal that exploiting mirrors´ various reflective properties yields a more robust, accurate, and simpler 3D position estimation approach than general-purpose stereo vision methods that use a linear approach or maximum-likelihood optimization. Our experiments show a root mean square (RMS) error of less than 2 mm in 3D space with only 20-point correspondences. For semiautomatic 3D motion tracking, we use an adaptive Kalman predictor and filter to improve stability and infer the occluded markers´ position. Our approach tracks more than 50 markers on a subject´s face and lips from 30-frame-per-second video clips. We´ve applied the facial motion parameters estimated from the proposed method to our facial animation system.