Sparsity based depth estimation and hole-filling algorithm for 2D to 3D video conversion

Depth image-based rendering (DIBR) requires to efficiently fill out holes in rendered image, and this has a great impact on the quality of 3D depth perception. In this paper, we propose a hole-filling method using depth estimation and image inpainting for high quality 3D video. In this algorithm, we first generate rendered image and its associated depth map. Then, we estimate the depths in generated depth map´s hole. Using these values, we fill out the hole with depth-aided image inpainting based on the sparsity of hole. This method enables the inpainting to refer similar background texture. The proposed algorithm also fills out the patches of hole based on an edge-preserving priority, so that background edges of the image texture can be reconstructed faithfully. Experimental results show that the proposed algorithm provides better objective and subjective quality than previous works.