Nonuniform sampling for image-based rendering: convergence of image, vision, and graphics

Recent convergence of image processing, computer vision, and computer graphics has resulted in an exciting research topic referred to as image-based rendering (IBR). Widely used in applications ranging from movie special effects (e.g., "the Matrix") to building virtual environments, IBR has become a critical tool for creating visually exciting content. With IBR, real-world scenes can be captured and rendered directly from images captured by cameras, eliminating the need for computationally expensive modeling of 3D geometry or surface reflectance, as is often done in traditional computer graphics. Various approaches to IBR have been proposed to render the scenes correctly and effectively given the captured images. We propose an active scene-capturing algorithm to efficiently capture the images. Based on the images that have been taken and the geometry information known so far, the algorithm intelligently determines where to pose the camera to capture the scene for best rendering performance. This results in a nonuniform but optimal set of captured images.