Audio-visual integration in multimodal communication

Author

Chen, Tsuhan ; Rao, Ram R.

Author_Institution

AT&T Bell Labs., Holmdel, NJ, USA

Volume

86

Issue

5

fYear

1998

fDate

5/1/1998 12:00:00 AM

Firstpage

837

Lastpage

852

Abstract

We review recent research that examines audio-visual integration in multimodal communication. The topics include bimodality in human speech, human and automated lip reading, facial animation, lip synchronization, joint audio-video coding, and bimodal speaker verification. We also study the enabling technologies for these research topics, including automatic facial-feature tracking and audio-to-visual mapping. Recent progress in audio-visual research shows that joint processing of audio and video provides advantages that are not available when the audio and video are processed independently

Keywords

audio coding; audio-visual systems; computer animation; multimedia communication; speaker recognition; synchronisation; video coding; voice communication; audio-to-visual mapping; audio-video coding; audio-visual integration; automatic facial-feature tracking; bimodal speaker verification; bimodality; facial animation; human speech; lip reading; lip synchronization; multimedia communication; multimodal communication; research; speech communication; video signal processing; Computer graphics; Facial animation; Humans; Image coding; Signal synthesis; Speech analysis; Speech synthesis; Video compression; Video signal processing; Videoconference;

fLanguage

English

Journal_Title

Proceedings of the IEEE

Publisher

ieee

ISSN

0018-9219

Type

jour

DOI

10.1109/5.664274

Filename

664274