A Web-Based Sign Language Translator Using 3D Video Processing

The American Sign Language (ASL) is used by hearing-impaired people in North America, as well as in other parts of the world to supplement indigenous sign language. A proof-of-concept ASL Translator has been designed and developed using 3D video processing techniques. Foreseeing its potential as a Web-based application, the Translator must have a portable input device to capture gestures and its cost must be kept low. The recently introduced Xbox Kinect is a versatile gesture input device and fits the low-cost requirement as well. 3D data of the joints of a user captured by the Kinect are analyzed and matched to a library of pre-recorded signs. The matched signs are then transcribed to word or phrase, and output to a suitable user interface. The implemented prototype works with excellent accuracy for a limited vocabulary. Using the Web and a server to archive the pre-recorded signs and to process recorded gesture via a motion capture device, there are many potential applications. The Translator can be utilized as an assistive tool for the hearing impaired to communicate or as a teaching tool for those who want to learn the sign language.