3D face recognition from multiple images: A shape-from-motion approach

In this article we explore the use of methodologies for 3D reconstruction from multiple images to recognize faces. We try to devise a strategy to tackle the problem of recognizing faces from images exhibiting strong pose (rotation and occlusion) and without prior knowledge (uncalibrated cameras, images from different sources). We do so by framing the recognition in the context of 3D structure from motion with missing data problems. In fact, recently, there has been a strong trend towards using 3D information to verify and recognize faces. However most of the state of the art works are developed over 3D sensors (3D range finders, stereo).Here we propose to do recognition by measuring the likelihood that one or more images were generated by a given 3D shape. In other words, given the subject´s 3D shape, we compute the required transformation to match the probe images and measure its deviation from rigidity. This process is not straightforward due to the strong pose: face points are not visible in all images so there is a ldquomissing datardquo problem intrinsic to the formulation. One key assumption is that all images (and shape) correspond to neutral expression. Though limited in scale due to lack of large databases, a set of tests demonstrate the adequacy and good performance of the approach.