Robust single view room structure segmentation in Manhattan-like environments from stereo vision

In this paper we propose a novel approach for the robust segmentation of room structure using Manhattan world assumption i.e. the frequently observed dominance of three mutually orthogonal vanishing directions in man-made environments. First, separate histograms are generated for the Cartesian major axis, i.e. X, Y and Z, on stereo data with an arbitrary roll, pitch and yaw rotation. Using the traditional Markov particle filters and minimal entropy as metric on the histograms, we are able to estimate the camera orientation with respect to orthogonal structure. Once the orientation is estimated we extract a hypotheses of the room structure by exploiting 2D histograms using mean shift clustering techniques as rough estimate for a pre-segmentation of voxels i.e. plane orientation and position. We apply superpixel over segmentation on the colour input to achieve a dense segmentation. The over segmentation and pre-segmented voxels are combined using graph-cuts for a not a-priori known number of final plane segments with a α-expansion graph cut variant proposed by Delong et al. with polynomial runtime. We show the robustness of our approach with respect to noise in real world data.