Title :
Audio-visual integration in multimodal communication
Author :
Chen, Tsuhan ; Rao, Ram R.
Author_Institution :
AT&T Bell Labs., Holmdel, NJ, USA
fDate :
5/1/1998 12:00:00 AM
Abstract :
We review recent research that examines audio-visual integration in multimodal communication. The topics include bimodality in human speech, human and automated lip reading, facial animation, lip synchronization, joint audio-video coding, and bimodal speaker verification. We also study the enabling technologies for these research topics, including automatic facial-feature tracking and audio-to-visual mapping. Recent progress in audio-visual research shows that joint processing of audio and video provides advantages that are not available when the audio and video are processed independently
Keywords :
audio coding; audio-visual systems; computer animation; multimedia communication; speaker recognition; synchronisation; video coding; voice communication; audio-to-visual mapping; audio-video coding; audio-visual integration; automatic facial-feature tracking; bimodal speaker verification; bimodality; facial animation; human speech; lip reading; lip synchronization; multimedia communication; multimodal communication; research; speech communication; video signal processing; Computer graphics; Facial animation; Humans; Image coding; Signal synthesis; Speech analysis; Speech synthesis; Video compression; Video signal processing; Videoconference;
Journal_Title :
Proceedings of the IEEE
