Hand Gesture Segmentation in Uncontrolled Environments with Partition Matrix and a Spotting Scheme Based on Hidden Conditional Random Fields

Hand gesture segmentation is the task of interpreting and spotting meaningful hand gestures from continuous hand gesture sequences with non-sign transitional hand movements. In real world scenarios, challenges from the unconstrained environments can largely affect the performance of gesture segmentation. In this paper, we propose a gesture spotting scheme which can detect and monitor all eligible hand candidates in the scene, and evaluate their movement trajectories with a novel method called Partition Matrix based on Hidden Conditional Random Fields. Our experimental results demonstrate that the proposed method can spot meaningful hand gestures from continuous gesture stream with 2-4 people randomly moving around in an uncontrolled background.