Projected pixel localization and artifact removal in captured images

Projector-camera systems are extensively used for various applications in computer vision, immersive environments, visual servoing, etc. Due to gaps between neighboring pixels on the projectorpsilas image plane and variations in scene depth, the image projected onto a scene shows pixelation and blurring artifacts. In certain vision and graphics applications, it is desirable that a high quality composition of the scene and the projected image, excluding the artifacts, is captured, while retaining the scene characteristics. Localization of projected pixels can also help in dense estimation of scene shape. In this paper, we address the problem of localizing each of the projected pixels from a captured scene and restoring the captured image so that the pixelation and blurring artifacts of the projector are removed.We improve the quality of the captured image further by virtualizing a high resolution projector. i.e., we modify the captured image as if the scene were illuminated with a high-resolution projector. We propose robust solutions to these problems and demonstrate their effectiveness on scenes of different complexities.