HMM-based transmodal mapping from audio speech to talking faces

Describes a transmodal mapping from audio speech to talking faces based on hidden Markov models (HMMs). If face movements are synthesized well enough for natural communication, a lot of benefits will be brought to human-machine communication. This paper describes an HMM-based speech-driven lip movement synthesis. The paper also describes its improvement by audio-visual joint estimation and its extension to talking face generation. The results of evaluation experiments show that the proposed method generates natural and accurate talking faces from audio speech inputs