Dense and Deformable Motion Extraction in Dynamic Scenes Based on Hierarchical MRF Optimization in RGB-D Images

We present a novel hierarchical MRFs optimization method for dense and deformable motion extraction in dynamic scenes. In particular, this hierarchical MRFs structure consists of two layers, the segmentation and the correspondence layer. Firstly, dynamic RGB-D foreground data is segmented through a pixel-level MRF in the segmentation layer. Subsequently, the extracted foreground data is transformed into a 3D point-level MRF in the correspondence layer. A new surface descriptor named deformable color and shape histogram is proposed. It is combined with photometric and geometric features to represent a deformable surface. Finally, the dynamic scene motion is retrieved from correspondences established in the image sequence. Discrete optimization schemes are used for the binary classification and multi-labeling problems. We provide an RGB-D dataset of dynamic scenes, which involves different motion patterns and surface properties of foreground objects. The effectiveness and efficiency of our proposed approach for high accurate foreground segmentation and motion extraction is validated in experiments.