Elevation-based MRF stereo implemented in real-time on a GPU

We describe a novel framework for calculating dense, accurate elevation maps from stereo, in which the height of each point in the scene is estimated relative to the ground plane. The key to our framework´s ability to estimate elevation accurately is an MRF formulation of stereo that directly represents elevation at each pixel instead of the usual disparity. By enforcing smoothness of elevation rather than disparity (using pairwise interactions in the MRF), the usual fronto-parallel bias is transformed into a horizontal (parallel to the ground) bias - a bias that is more appropriate for scenes characterized by a dominant ground plane viewed from an angle. This horizontal bias amounts to a more informative prior for such scenes, which results in more accurate surface reconstruction, with sub-pixel accuracy. We apply this framework to the problem of finding small obstacles, such as curbs and other small deviations from the ground plane, a few meters in front of a vehicle (such as a wheelchair or robot) that are missed by standard real-time correlation stereo algorithms. We demonstrate a real-time implementation of our framework on a GPU (we have made the code publicly available), which processes a 640 × 480 stereo image pair in 160 ms using either our elevation model or a standard disparity-based model (with 32 elevation or disparity levels), and describe experimental results.