Model-based multiple view reconstruction of people

This paper presents a framework to reconstruct a scene captured in multiple camera views based on a prior model of the scene geometry. The framework is applied to the capture of animated models of people. A multiple camera studio is used to simultaneously capture a moving person from multiple viewpoints. A humanoid computer graphics model is animated to match the pose at each time frame. Constrained optimisation is then used to recover the multiple view correspondence from silhouette, stereo and feature cues, updating the geometry and appearance of the model. The key contribution of this paper is a model-based computer vision framework for the reconstruction of shape and appearance from multiple views. This is compared to current model-free approaches for multiple view scene capture. The technique demonstrates improved scene reconstruction in the presence of visual ambiguities and provides the means to capture a dynamic scene with a consistent model that is instrumented with an animation structure to edit the scene dynamics or to synthesise new content.