Synthesizing pose and lighting variation from object motion

We present a novel method to synthesize images of a 3D object in arbitrary poses illuminated from arbitrary directions, given a few images of the object in unknown motion under static lighting. Our scheme is underpinned by the notion of illumination image basis which spans an image space of arbitrary lighting, and we propose to generate it by a recursive search for the correspondence between input images and by subsequent realignment of their pixels. Using the 3D surface of the object that also becomes available in this procedure, we synthesize images of the object in arbitrary poses while arbitrarily varying the direction of lighting by combination of the illumination basis images. The effectiveness of the entire algorithm is shown through experiments.