A 2D+3D face authentication system robust under pose and illumination variations

A complete face authentication system integrating 2D color and 3D depth images is described in this paper. Depth information, acquired by a novel low-cost 3D and color sensor based on the structured light approach, is used for robust face detection, localization and 3D pose estimation. To cope with illumination and pose variations, 3D information is used for the normalization of the input images. Illumination compensation exploits depth data to recover the illumination of the scene and relight the image under frontal lighting. The performance of the proposed system is tested on a face database of more than 3,000 images, in conditions similar to those encountered in real-world applications. Experimental results show that when normalized images, depicting upright orientation and frontal lighting, are used for authentication, significantly lower error rates are achieved. Moreover, the combination of color and depth data results in increased authentication accuracy, compared to the use of each modality alone.