Accurate surface description from binocular stereo

The authors present a stereo vision system in which they attempt to achieve robustness with respect to scene characteristics, from textured outdoor scenes to environments composed of highly regular man-made objects. The system offers the advantages of both area-based (dense map) and feature-based (accurate disparity) processing by combining them whenever possible. The authors are able to geneate a disparity map that is sufficiently accurate to allow them to detect depth and surface orientation discontinuities, provided that the resolution is fine enough. They use an area-based cross-correlation, along with an ordering constraint and a weak surface smoothness assumption to produce an initial disparity map. Unlike other approaches, however, a match is accepted only if both views agree on a correlation peak and this peak is strong enough. This disparity map is a blurred version of the true one, however, because of the smoothing inherent in the correlation. The problem is most acute at C₀ (depth) and C₁ (crease) discontinuities but can be mitigated by introducing the edge information: the disparity map is adaptively smoothed subject to the constraint that the disparity at edges is fixed