A lip reading application on MS Kinect camera

Author

Yargic, Alper ; Dogan, M.

Author_Institution

Comput. Eng. Dept., Anadolu Univ., Eskisehir, Turkey

fYear

2013

fDate

19-21 June 2013

Firstpage

1

Lastpage

5

Abstract

Hearing-impaired people can read lips and lip reading applications may help them to improve their lip imitation skills. Speech of normal people can be recognized by even cellular phones but lip reading systems using only visual features remain important for hearing-impaired people. This paper aims to develop an application using MS Kinect camera to recognize Turkish color names to be used in the education of hearing-impaired children. Predefined lip points are located with depth information by the MS Kinect Face Tracking SDK. Words are segmented from the speech and the angles between the lip points are used as features to classify the words. Angles are computed using the 3D coordinates of the lip points. The KNN classifier is used to classify the words with Manhattan and Euclidean distances and the best feature vectors are tried to be found. As a result, the isolated words are classified with the success rate of 78.22%.

Keywords

face recognition; feature extraction; handicapped aids; image classification; image colour analysis; object tracking; speech recognition; 3D coordinates; Euclidean distances; KNN classifier; MS Kinect Face Tracking SDK; MS Kinect camera; Manhattan distances; Turkish color name recognition; cellular phones; depth information; feature vectors; hearing-impaired children education; hearing-impaired people; lip imitation skills improvement; lip reading application; lip reading systems; normal people speech recognition; predefined lip point location; visual features; word classification; word segmentation; Cameras; Face; Mouth; Speech; Speech recognition; Tracking; Visualization; 3D face tracking; K-nearest neighbor classifier; Lip reading; MS Kinect camera; lip activation; lip motion detection;

fLanguage

English

Publisher

ieee

Conference_Titel

Innovations in Intelligent Systems and Applications (INISTA), 2013 IEEE International Symposium on

Conference_Location

Albena

Print_ISBN

978-1-4799-0659-8

Type

conf

DOI

10.1109/INISTA.2013.6577656

Filename

6577656