Surface electromyography and acceleration based Sign Language Recognition using Hidden Conditional Random Fields

Sign Language Recognition has numerous applications, such as building a platform for the communication between the deaf and the hearing world. In this paper, Hidden Conditional Random Field (HCRF), a novel probabilistic model which has already been used in the area of speech and image recognition, was proposed for Sign Language Recognition (SLR) based on surface electromyography (sEMG) and acceleration (ACC) signals. Because of its latent and discriminative property, HCRF, as a branch of CRF models, was selected for this SLR task. In the proposed method, after the periods of data acquisition, data segmentation, feature extraction, and preliminary recognition on the decision-tree level, HCRF was utilized in the bottom layer to classify an observation sequence into a specific class. Experiments conducted on five subjects and 120 high-frequency used Chinese sign language subwords obtained 91.51% averaged recognition accuracy. This result demonstrated that HCRF is feasible and effective for the sEMG and ACC based Sign Language Recognition.