Piecewise Planar Decomposition of 3D Point Clouds Obtained from Multiple Static RGB-D Cameras

In this paper, we address the problem of segmenting a 3D point cloud obtained from several RGB-D cameras into a set of 3D piecewise planar regions. This is a fundamental problem in computer vision, whose solution is helpful for further scene analysis, such as support inference and object localisation. In existing planar segmentation approaches for point clouds, the point cloud originates from a single RGB-D view. There is however a growing interest to monitor environments with computer vision setups that contain a set of calibrated 3D cameras located around the scene. To fully exploit the multi-view aspect of such setups, we propose in this paper a novel approach to perform the planar piecewise segmentation directly in 3D. This approach, called Voxel-MRF (V-MRF), is based on discrete 3D Markov random fields, whose nodes correspond to scene voxels and whose labels represent 3D planes. The voxelization of the scene permits to cope with noisy depth measurements, while the MRF formulation provides a natural handling of the 3D spatial constraints during the optimisation. The approach results in a decomposition of the scene into a set of 3D planar patches. A by-product of the method is also a joint planar segmentation of the original images into planar regions with consistent labels across the views. We demonstrate the advantages of our approach using a benchmark dataset of objects with known geometry. We also present qualitative results on challenging data acquired by a multi-camera system installed in two operating rooms.