Profile View Lip Reading

Author

Kumar, Kush ; Tsuhan Chen ; Stern, Richard M.

Author_Institution

Dept. of Electr. & Comput. Eng., Carnegie Mellon Univ., Pittsburgh, PA, USA

Volume

4

fYear

2007

fDate

15-20 April 2007

Abstract

In this paper, we introduce profile view (PV) lip reading, a scheme for speaker-dependent isolated word speech recognition. We provide historic motivation for PV from the importance of profile images in facial animation for lip reading, and we present feature extraction schemes for PV as well as for the traditional frontal view (FV) approach. We compare lip reading results for PV and FV, which demonstrate a significant improvement for PV over FV. We show improvement in speech recognition with the integration of audio and visual features. We also found it advantageous to process the visual features over a longer duration than the duration marked by the endpoints of the speech utterance.

Keywords

feature extraction; image processing; speech recognition; facial animation; feature extraction; frontal view approach; profile images; profile view lip reading; speaker-dependent isolated word speech recognition; Audio databases; Cellular phones; Discrete cosine transforms; Feature extraction; Humans; Image motion analysis; Speech recognition; Speech synthesis; Visual databases; Visualization; Audiovisual speech recognition; Profile view; Speechreading; Visual feature extraction;

fLanguage

English

Publisher

ieee

Conference_Titel

Acoustics, Speech and Signal Processing, 2007. ICASSP 2007. IEEE International Conference on

Conference_Location

Honolulu, HI

ISSN

1520-6149

Print_ISBN

1-4244-0727-3

Type

conf

DOI

10.1109/ICASSP.2007.366941

Filename

4218129