A DTW and Fourier Descriptor based approach for Indian Sign Language recognition

Humans with hearing disabilities are highly dependent on non-verbal forms of communication involving hand gestures. A gesture recognition system capable of transforming gestures to other forms of communication can bring about a significant improvement in the quality of life of such human beings. A novel approach making use of Fourier Descriptors for shape extraction followed by Dynamic Time Warping for sequence matching in order to recognize gestures representing English alphabets present in the Indian Sign Language (ISL) has been proposed in this paper. A feature vector constituting the top 220 coefficients of the Fourier Descriptor is calculated for all images present in the training and test dataset. The test sequences are matched with the training sequences using Dynamic time Warping algorithm in order to calculate the similarity between the test image and all the images of the training dataset. The results are stored in the increasing order of the cost between the two sequences which is a measure of similarity between the two sequences. K-nearest neighbor algorithm is then applied to a cost vector containing 15 gestures having the minimum cost with the test image to recognize the most dominant gesture. A data set constituting of 338 images is used for training the algorithm for classifying 26 different characters. The algorithm was found to have an accuracy of 96.15 % when tested on a dataset of 78 images.