Hand tracking based on the combination of 2D and 3D model in gaze-directed video

This paper investigates model based hand tracking in gaze directed video which contains everyday manipulation activity of human in kitchen environment. The video is recorded by a gaze-directed camera, which can actively directs at the visual attention area from the person who wears the camera. Here we present a method based on the combination of 2D and 3D hand model, which can estimate the position of hand in image accurately and the pose of hand in 3D roughly. The method uses 2D model tracking result to initialize and predict 3D tracking, which saves the number of particles and makes it possible for local configuration adapting. To evaluate our re sult, we try our algorithm on several pieces of video both from normal camera and gaze-directed camera. The error ratio of the distance between the ground truth and tracking result is used as an objective measurement for evaluating our method. Trajectory of hand movement and results of projected model for every frame show that our method is effective and makes a good foundation for future recognition and analysis.