Occlusion filling in stereo: Theory and experiments

A number of stereo matching algorithms have been developed in the last few years, which also have successfully detected occlusions in stereo images. These algorithms typically fall short of a systematic study of occlusions; they predominantly emphasize matching and regard occlusion filling as a secondary operation. Filling occlusions, however, is useful in many applications such as image-based rendering where 3D models are desired to be as complete as possible. In this paper, we study occlusions in a systematic way and propose two algorithms to fill occlusions reliably by applying statistical modeling, visibility constraints, and scene constraints. We introduce a probabilistic, model-based filling order of the occluded points to maintain consistency in filling. Furthermore, we show how an ambiguity in the interpolation of the disparity value of an occluded point can safely be avoided using color homogeneity when the point’s neighborhood consists of multiple scene surfaces. We perform a comparative study and show that statistically, the new algorithms deliver good quality results compared to existing algorithms.