Hand and object segmentation from RGB-D images for interaction with planar surfaces

We introduce a new approach for hand and object segmentation using RGB-D cameras suitable for gesture-based Human-Computer Interfaces (HCIs) that involve an interaction plane. The technique consists of detecting the interaction plane using a temporally coherent version of RANSAC, followed by segmenting off-plane objects using a markers-based watershed transform with an energy function that combines depth and chromaticity gradients. Experimental results show that our approach can segment the hand even when it is very close to the interaction plane, unlike traditional approaches based on distance thresholds.