Temporal Alignment of 3D Video Sequences Using Shape and Appearance

In this paper we present a method to automatically introduce temporal consistency into a sequence of meshes reconstructed from multi-view video. We combine a number of advantages of previous work to produce a robust algorithm. Our approach uses SIFT features in each camera view combined with geometric information to produce accurate frame to frame correspondence. Coupling this with both volumetric and surface laplacian deformation techniques we are able to deform a well reconstructed frame across the 3D video sequence. Although the approach is sequential the combination of appearance and shape matching allows recovery from errors in previous frames or surface reconstruction.