3D visual pronunciation of Mandarine Chinese for language learning

This paper proposes a text-driven 3D visual pronunciation system of Mandarin Chinese. Firstly, an articulatory speech corpus is collected through Electro-Magnetic Articulography, and then articulatory model and acoustic model are trained by multi-stream Hidden Semi-Markov Model together; secondly, based on an accurate anatomical 3D articulatory mesh model constructed by Magnetic Resonance Imaging, the Hidden Semi-Markov Model synthesized result is combined with articulatory anatomy to produce realistic articulatory animation synchronized with speech. The experimental results show the effectiveness of the system for instructing language learners to articulate.