FPGA-based real-time disparity computation and object location

This paper describes an FPGA-based system capable of computing the distance of objects in a scene to two stereo cameras, and use that information to isolate objects in the foreground. For this purpose, four disparity maps are generated in real time, according to different similarity metrics and sweep directions, and then merged into a single foreground-versus-background bitmap. Our main contribution is a custom-built hardware architecture for the disparity map calculation, and an optional post-processing stage that coarsens the output to improve resilience against spurious results. The system was described in Verilog, and a prototype implemented on a Xilinx Virtex-II Pro FPGA proved capable of processing 640×480 black-and-white images at a maximum frame rate of 40 fps, using 3×3 matching windows and detecting disparities of up to 135 pixels.