Visual speech synthesis based on Chinese dynamic visemes

In order to realize realistic visual speech synthesis, a visual speech synthesis method based on Chinese dynamic visemes is proposed. With mouth feature parameters of Chinese static visemes, consonants and vowels are classified using clustering algorithm. According to Chinese pronunciation characters, we can get 40 basic dynamic visemes by combining consonant types and vowel types. With these dynamic visemes and corresponding phonemes, two-layer hidden Markov model (HMM) is built up and trained. Experimental results show that, for speeches which are chosen randomly, the synthetic visual speech is smooth and realistic.