Image-based rendering of complex scenes from a multi-camera rig

Image-based rendering is a method to synthesise novel views from a set of given real images. Two methods to extrapolate novel views of complex scenes with occlusions and large depth discontinuities from images of a moving uncalibrated multi-camera rig are described. The real camera viewpoints are calibrated from the image data and dense depth maps are estimated for each real view. Novel views are synthesised from this representation with view-dependent image-based rendering techniques at interactive rates. Since the 3D scene geometry is available in this approach, it is well suited for mixed reality applications where synthetic 3D objects are seamlessly embedded in the novel view.