Title :
Arabic sign language recognition using the leap motion controller
Author :
Mohandes, M. ; Aliyu, S. ; Deriche, M.
Author_Institution :
Electr. Eng. Dept., King Fahd Univ. of Pet. & Miner., Dhahran, Saudi Arabia
Abstract :
Sign language is important for facilitating communication between hearing impaired and the rest of society. Two approaches have traditionally been used in the literature: image-based and sensor-based systems. Sensor-based systems require the user to wear electronic gloves while performing the signs. The glove includes a number of sensors detecting different hand and finger articulations. Image-based systems use camera(s) to acquire a sequence of images of the hand. Each of the two approaches has its own disadvantages. The sensor-based method is not natural as the user must wear a cumbersome instrument while the imagebased system requires specific background and environmental conditions to achieve high accuracy. In this paper, we propose a new approach for Arabic Sign Language Recognition (ArSLR) which involves the use of the recently introduced Leap Motion Controller (LMC). This device detects and tracks the hand and fingers to provide position and motion information. We propose to use the LMC as a backbone of the ArSLR system. In addition to data acquisition, the system includes a preprocessing stage, a feature extraction stage, and a classification stage. We compare the performance of Multilayer Perceptron (MLP) neural networks with the Nave Bayes classifier. Using the proposed system on the Arabic sign alphabets gives 98% classification accuracy with the Nave Bayes classifier and more than 99% using the MLP.
Keywords :
data gloves; feature extraction; image classification; image sequences; multilayer perceptrons; sign language recognition; ArSLR; Arabic sign alphabets; Arabic sign language recognition; LMC; MLP neural networks; classification accuracy; classification stage; electronic gloves; feature extraction stage; finger articulations; hand articulations; image sequences; image-based systems; leap motion controller; motion information; multilayer perceptron; naive Bayes classifier; position information; preprocessing stage; sensor-based systems; Accuracy; Assistive technology; Equations; Feature extraction; Gesture recognition; Neurons; Vectors; Arabic sign langauge recognition; electronic glove; finger articulation; image-based system; leap motion controller;
Conference_Titel :
Industrial Electronics (ISIE), 2014 IEEE 23rd International Symposium on
Conference_Location :
Istanbul
DOI :
10.1109/ISIE.2014.6864742