Visual hand motion capture for guiding a dexterous hand

Visual hand gesture based interfaces have been widely used for navigation of virtual environments (VE) and control of a robot arm in robotic systems. In fact, hand movement has abundant powers of expression with complex finger joint motions. In this paper, full-degree-of-freedom (DOF) hand motion is captured in a vision-based mode. It makes human-robot-interaction more intuitive and control of dexterous hand feasible. However, hand motion capture is a high dimensional search problem with serious ambiguity. A learning integrated with optimization approach in 3-D model based framework is introduced for pose estimation and a motion-tracking scheme with GA-based particle filter (PF) is described as a solution of high-dimensional and multi-modal search problem. Test experiment results show strong capabilities of such a hand motion capture system, especially in deal with high dimensional search problem.