Patch-sweeping with robust prior for high precision depth estimation in real-time systems

This paper presents a novel real-time approach for robust high precision and high quality depth estimation. It extends recent work on real-time Patch-Sweeping by combining the advantages of a robust hybrid stereo-based disparity estimator with the high accuracy of the Patch-Sweeping approach. It over- comes limitations of the existing Patch-Sweep approach, such as limited search range. Further, it implicitly benefits from the high robustness as well as time consistency of the disparity estimator. The presented overall algorithmic system concept introduces a powerful alternative to traditional real-time depth estimation approaches. Additionally, the proposed algorithmic structures allow a high degree of parallelization. Based on this, the computational effort could be efficiently balanced between GPU and CPU processing. The target platform of the proposed algorithmic chain is a real-time immersive 3D video communication system which requires highly accurate 3D estimation results for a high quality virtual eye contact generation.