Audiovisual speech processing

Author

Chen, Tsuhan

Author_Institution

Dept. of Electr. & Comput. Eng., Carnegie Mellon Univ., Pittsburgh, PA, USA

Volume

18

Issue

1

fYear

2001

fDate

1/1/2001 12:00:00 AM

Firstpage

9

Lastpage

21

Abstract

We have reported activities in audiovisual speech processing, with emphasis on lip reading and lip synchronization. These research results have shown that, with lip reading, it is possible to enhance the reliability of audio speech recognition, which may result in a computer that can truly understand the user via hand-free natural spoken language even in a very noisy environments. Similarly, with lip synchronization, it is possible to render realistic talking heads with lip movements synchronized with the voice, which is very useful for human-computer interactions. We envision that in the near future, advancement in audiovisual speech processing will greatly increase the usability of computers. Once that happens, the cameras and the microphone may replace the keyboard and the mouse as better mechanisms for human-computer interaction

Keywords

gesture recognition; natural language interfaces; rendering (computer graphics); speech recognition; speech-based user interfaces; synchronisation; audio speech recognition; audiovisual speech processing; hand-free natural spoken language; human-computer interactions; lip movements; lip reading; lip synchronization; noisy environment; realistic talking heads; reliability; rendering; Acoustic waves; Computer aided instruction; Face detection; Facial muscles; Humans; Lips; Loudspeakers; Productivity; Speech processing; Tongue;

fLanguage

English

Journal_Title

Signal Processing Magazine, IEEE

Publisher

ieee

ISSN

1053-5888

Type

jour

DOI

10.1109/79.911195

Filename

911195