Deformable Surface Augmentation in Spite of Self-Occlusions

The augmentation problem for images of a deforming surface has been studied since recently. The surface is usually assumed not to be self-occluding. Two dimensional deformation estimation in the presence of self-occlusions is very challenging. This paper proposes a specific framework explicitly modeling self-occlusions for augmented reality applications. The basic idea is to detect self-occlusions as warp shrinkage areas. Deformations are initially estimated via direct non-rigid image registration. Temporal smoothness is then used to refine the warps and the image are augmented. Experimental results on several challenging datasets show that our approach convincingly augments self-occluded surfaces. Associated videos are available on the first author´s Web homepage.