A Natural Chinese speech Driven Mouth Animation System

Distinguish with phoneme based human mouth animation researches, this paper presents a novel natural speech driven mouth animation approach. To recognize the mouth shapes sequence from continuous speech, the context-dependent viseme (triseme) modeling technique is employed for acquiring the trisemic HMMs. To obtain the robust model parameters with the limited training data, the states tying procedure is introduced. Considering the compatibility and ambiguity issues, the visemic questions which assigned in the leaf nodes of decision tree are generated that based on the training data. With the modeled HMM parameters, the Viterbi beam searching algorithm is applied to time align the trisemic sequences. Mapping the recognized trisemes to the corresponding MPEG-4 FAPs represented mouth shapes, the speaking mouth can be finally animated through a smoothing process. The experimental results demonstrate that the recognition accuracy is applicable and also the recognizing and aligning speed is acceptable in human vision range.