Vision-based continuous sign language recognition using product HMM

This paper introduces a vision-based continuous sign language recognition (CSR) system. This CSR system can differentiate the signs in vocabulary and the non-signs. First, the continuous sign language is segmented into isolated sign segments. Then, the sign segment which can be interpreted by Product-HMMs (pHMM) is a sign, otherwise it is a non-sign. In the experiments, we test 40 signs from Taiwanese Sign Language. Our system achieves a good performance of sign recognition accuracy of 94.04%. We also test three continuous sign language which consist of 18~23 different signs. The experimental results show that the average sign recognition recall rate is 74.5% and precision rate is 89%.