Title :
Application of affine-invariant Fourier descriptors to lipreading for audio-visual speech recognition
Author :
Gurbuz, Sabri ; Tufekci, Zekeriya ; Patterson, Eric ; Gowdy, John N.
Author_Institution :
Dept. of Electr. & Comput. Eng., Clemson Univ., SC, USA
Abstract :
Focuses on an affine-invariant lipreading method, and its optimal combination with an audio subsystem to implement an audio-visual automatic speech recognition (AV-ASR) system. The lipreading method is based on outer lip contour description which is transformed to the Fourier domain and normalized there to eliminate dependencies on the affine transformation (translation, rotation, scaling, and shear) and on the starting point. The optimal combination algorithm incorporates a signal-to-noise ratio (SNR) based weight selection rule which leads to a more accurate global likelihood ratio test. Experimental results are presented for an isolated word recognition task for eight different noise types from the NOISEX data base for several SNR values
Keywords :
Fourier transforms; audio signal processing; feature extraction; hidden Markov models; speech recognition; video signal processing; NOISEX database; SNR based weight selection rule; affine-invariant Fourier descriptors; audio-visual speech recognition; isolated word recognition; lipreading; outer lip contour description; Acoustic noise; Acoustic testing; Application software; Automatic speech recognition; Degradation; Feature extraction; Hidden Markov models; Humans; Signal to noise ratio; Speech recognition;
Conference_Titel :
Acoustics, Speech, and Signal Processing, 2001. Proceedings. (ICASSP '01). 2001 IEEE International Conference on
Conference_Location :
Salt Lake City, UT
Print_ISBN :
0-7803-7041-4
DOI :
10.1109/ICASSP.2001.940796