A Regularized Technique for Real-time Interposition of 3D Objects in Augmented Reality

This article presents an innovative solution to the interposition problem in augmented reality in natural, unstructured environments when the real data to augment consists of video streams provided by a stereoscopic pair of cameras. The realistic and physically-consistent embedding of a 3D virtual object into a real scene is accomplished by first considering this object as a set of independent 3D points. Rather than relying on the 3D reconstruction of the scene, our method exploits results of local stereo matching to decide whether each virtual point should be visible or not for either camera. The second phase of our method uses the spatiotemporal piecewise-constant nature of the visibility property of virtual points to build a regularized visibility map for the virtual object as a whole