A Real-Time Hand Pose Estimation System with Retrieval

In this paper, we propose a real-time system of hand pose estimation that infers the hand pose and shows the finger positions in each frame. The system is designed through data driven methodology. For the system to perform in real-time, we employ a retrieval method based on an inverted-file index with edge-based descriptors. To strengthen the discriminability, we combine a robust orientation assignment method with the descriptors. A novel alignment method is designed to transfer pose information from the retrieval results to the recognized image. To refine the retrieval results in the measurement of pose similarity, a mixed criterion that considers both outline deviation and descriptor matching is also proposed. The proposed system is quite scalable with respect to poses, since unobserved poses can be added freely. To evaluate the retrieval performance, we apply criteria that assess about the deviation of the corresponding positions of the query pose from the retrieval results. The evaluation is carried out on a retrieval dataset with manually marked images. Our proposed method effectively estimates pose at 20 frames per second, as demonstrated in our experiments on both real and publicly available datasets.