Patch-based stereo in a general binocular viewing geometry

This paper presents a one-stage stereo algorithm that yields 3D planar surface patches directly from matching image patch intensity information. The method allows an arbitrary rotation and translation between the cameras; it is not limited to parallel-axis, narrow-baseline, or vergent geometries. The key to the approach is to match image patches that have positions, shapes, sizes, orientations, and samplings consistent with a hypothesized surface patch and with each other. The match error then reflects only the mismatch of patch contents and not the mismatch of patch geometries or samplings. The algorithm is quantitatively evaluated against ground truth on real images with difficult viewing geometries, and demonstrates an average accuracy of about 1% in estimating surface depths and 10° in estimating surface normals