Dimensionality of illumination in appearance matching

Appearance matching was recently demonstrated as a robust and efficient approach to 3D object recognition and pose estimation. Each object is represented as a continuous appearance manifold in a low-dimensional subspace parametrized by object pose and illumination direction. Here, the structural properties of appearance manifolds are analyzed with the aim of making appearance representation efficient in off-line computation, storage requirements, and online recognition time. In particular, the effect of illumination on the structure of the appearance manifold is studied. It is shown that for an ideal diffused surface of arbitrary texture, the appearance manifold is linear and three dimensional. This enables the construction of the entire illumination manifold from just three images of the object taken using linearly independent light sources. This result is shown to hold even for illumination by multiple light sources and for concave surfaces that exhibit inter-reflections. Finally, a simple but efficient algorithm is presented that uses just three manifold points for recognizing images taken under novel illuminations