Non-parametric and light-field deformable models

Statistical shape-and-texture appearance models use image morphing to define a rich, compact representation of object appearance. They are useful in a variety of applications including object recognition, tracking and segmentation. These techniques, however, have been limited to objects with Lambertian surface reflectance, simple geometry and topology. In this work, we present new shape-and-texture appearance models that overcome these limitations. In the first part of our work, we develop a 4D shape-and-texture appearance model, built using light-fields. This model is capable of representing objects with complex surface reflectance and geometry. We demonstrate our light-field appearance model using 50 light-fields of the human head captured from a real-time camera array. Next, we present a non-parametric appearance model of the shape and texture of objects whose appearance manifolds exhibit a varying topology, e.g., have holes. We demonstrate this model using 2D mouth images of speaking people. In our experiments, we evaluate the performance of each method and provide a comparison with conventional, linear single- and multi-view deformable models.