Depth estimation from stereo images using sparsity

In this paper, we propose a new method for correcting the depth image, which was obtained by applying a matching scheme to stereo images and often includes parts with large error, based on the sparsity of depth image. When the depth image is obtained by a stereo matching, a small pixel correspondence error causes large estimation errors in the depth image. Also, original depth image can be considered to have sparsity the same as many natural signals without noise. So we correct the depth image that was obtained by the stereo matching, based on the sparsity of the original depth image. First, the depth image parts with a higher probability of containing large estimation errors are selected as the areas in which the depth has relatively large difference from that which was obtained by applying the median filter to the estimated depth image. Second, the depth image is applied with the inpainting procedure based on the data sparsity [1] as shown in Fig. 1 in which the data of the selected area are treated as being lost. In particular the depth image in a region, which corresponds to an object in 3-D space, is wavelet transformed by SA-DWT (Shape-Adaptive Discrete Wavelet Transform). The smaller wavelet coefficients are truncated to zero with a threshold procedure. With decreasing threshold value, the wavelet transform, smaller coefficient zeroing, and the inverse wavelet transform processes are repeated until the processed depth image is converged. Experiments show that the proposed method is able to remove large errors in the depth image which had been obtained by the stereo matching scheme.