Face recognition under variable lighting using harmonic image exemplars

We propose a new approach for face recognition under arbitrary illumination conditions, which requires only one training image per subject (if there is no pose variation) and no 3D shape information. Our method is based on the result of Basri and Jacobs (2001), which demonstrated that the set of images of a convex Lambertian object obtained under a wide variety of lighting conditions can be approximated accurately by a low-dimensional linear subspace. In this paper, we show that we can recover basis images spanning this space from just one image taken under arbitrary illumination conditions. First, using a bootstrap set consisting of 3D face models, we compute a statistical model for each basis image. During training, given a novel face image under arbitrary illumination, we recover a set of images for this face. We prove that these images are the set of basis images with maximum probability. During testing, we recognize the face for which there exists a weighted combination of basis images that is the closest to the test face image. We provide a series of experiments that achieve high recognition rates, under a wide range of illumination conditions, including multiple sources of illumination. Our method achieves comparable levels of accuracy with methods that have much more onerous training data requirements.