Two stage Real-Time stereo correspondence algorithm and FPGA architecture using a modified Generalized Hough transform

We present a two stage approach for the task of calculating the stereo correspondence by processing a stereo image pair. In the first stage, coarse disparities for line segments are found using a modified Generalized Hough transform approach and absolute differences. In the second stage, the coarse disparities and its neighbors are evaluated and refined with a local window based Census transform. Finally, the dense disparity map is filtered with a 3×3 Median filter to remove outliers. Additionally, we propose a parallel and pipelined FPGA design, processing one pair of pixels and delivering one disparity per clock cycle. We evaluated the disparity map quality with Middlebury [1] stereo datasets. Our approach reaches qualities which are comparable to recent hardware implementations but outperforms them in terms of hardware resource utilization and data throughput. For the highest quality configuration, a frame rate of 270 frames per second is achieved for Middlebury sized images of 450×375 pixels with a disparity range of D=65.