Breaking the ‘Ton’: Achieving 1% depth accuracy from stereo in real time

Stereo correspondence algorithms are computationally complex making real-time performance difficult on conventional hardware. Large disparity ranges are key to depth accuracy but increase the computation time: recently, we have achieved a disparity range of 128 enabling 1% depth accuracy in a system processing high resolution (1 Mpixel) images at 30 fps. The pixel throughput rate requires careful pipelining of the entire computation, which includes a fast distortion removal and rectification module attached to each camera followed by the correspondence module, which uses GimeFfarb´s SDPS algorithm. This algorithm has a small, compact hardware realization, permitting many copies computing in parallel to achieve a disparity range of 0-127. We describe the processing pipeline that was needed to handle the high pixel throughput. A key benefit of SDPS is that occlusion maps, clearly outlining distinct objects in scenes, are produced for no additional effort.