RGB-D scene segmentation with Conditional Random Field

Segmentation of a scene to the part made is a challenging work. In this paper a graphical model is used for this task. The methods based on geometrical derivatives such as curvature and normal often haven´t good result in segmentation of geometrically-complex architecture and lead to over-segmentation and even failure. Proposed method for segmentation contains two steps. At first region growing based on curvature, normal and color is used for growing region. This segmented cloud is used for unary potential in graphical model. Fully connected graph for Conditional Random Field with Gaussian kernel for pair wise potentials is used for correcting this segmentation. Gaussian kernels are based on appearance, smoothness and surface. This leads to high computational complexity since the model is fully connected and in every step of message passing needs to compute this Gaussian kernel between each node with all of the others node. Efficient Inference with Permutohedral high dimensional Lattice is used for doing this computation with high speed. This method is tested on challenging NYU depth 1 dataset with complicated geometry and results shows that the scene can segment to the part made it with high accuracy.