Chinese sign language recognition with 3D hand motion trajectories and depth images

An important part for sign language expression is hand shape, and the 3D hand motion trajectories also contain abundant information to interpret the meaning of sign language. In this paper, a novel feature descriptor is proposed for sign language recognition, the hand shape features extracted from the depth images and spherical coordinate (SPC) feature extracted from the 3D hand motion trajectories combine to make up the final feature representation. The new representation not only incorporates both the spatial and temporal information to depict the kinematic connectivity among hand, wrist and elbow for recognition effectively but also avoids the interference of the illumination change and cluttered background compared with other methods. Meanwhile, our self-built dataset includes 320 instances to evaluate the effectiveness of our combining feature. In experiments with the dataset and different feature representation, the superior performance of Extreme Learning Machine (ELM) is tested, compared with Support Vector Machine (SVM).