Real-time hand tracking on depth images

Hand tracking is a fundamental task in a gesture recognition system. Most previous works tracked the hand position on color images and relied heavily on skin color information. However, color information is very vulnerable to lighting variations and skin color varies across difference human races. Furthermore, one can not effectively discriminate faces or other skin-color-like objects from hands when using skin color detection. In this paper, we propose a hand tracking algorithm that uses depth images only, and also a hand click detection method to initialize the hand tracking automatically. We show that depth images suffice and are advantageous to real-time hand tracking. A region growing technique is applied to segment the hand region on depth images. Then a mean-shift based algorithm accurately locates the hand center in the segmented hand region. The experimental results show that the proposed tracking algorithm runs at 300+ FPS, and the average error of the tracked 3D hand positions is less than 1 centimeter. The proposed method enables a plethora of potential applications to natural Human-Computer Interaction (HCI), and is adequate for embedded systems of consumer electronics because of its low complexity and low bandwidth requirement.