Vision-based continuous sign language recognition using product HMM

Author

Yu, Shin-Han ; Huang, Chung-Lin ; Hsu, Shih-Chung ; Lin, Hung-Wei ; Wang, Hau-Wei

Author_Institution

Dept. of Electr. Eng., Nat. Tsing-Hua Univ., Hsinchu, Taiwan

fYear

2011

fDate

28-28 Nov. 2011

Firstpage

510

Lastpage

514

Abstract

This paper introduces a vision-based continuous sign language recognition (CSR) system. This CSR system can differentiate the signs in vocabulary and the non-signs. First, the continuous sign language is segmented into isolated sign segments. Then, the sign segment which can be interpreted by Product-HMMs (pHMM) is a sign, otherwise it is a non-sign. In the experiments, we test 40 signs from Taiwanese Sign Language. Our system achieves a good performance of sign recognition accuracy of 94.04%. We also test three continuous sign language which consist of 18~23 different signs. The experimental results show that the average sign recognition recall rate is 74.5% and precision rate is 89%.

Keywords

computer vision; gesture recognition; hidden Markov models; image segmentation; natural language processing; vocabulary; CSR system; Taiwanese sgn lnguage; average sign recognition recall rate; isolated sign segment; nonsigns; pHMM; product HMM; sign recognition accuracy; vocabulary; vsion-based continuous sign language recognition; Accuracy; Feature extraction; Handicapped aids; Hidden Markov models; Image segmentation; Support vector machines; Tracking; Continuous Sign Language Recognition (CSR); Movement epenthesis (ME); Product-HMM; support vector machine (SVM);

fLanguage

English

Publisher

ieee

Conference_Titel

Pattern Recognition (ACPR), 2011 First Asian Conference on

Conference_Location

Beijing

Print_ISBN

978-1-4577-0122-1

Type

conf

DOI

10.1109/ACPR.2011.6166631

Filename

6166631