Study on incorporating tone into speech recognition of Vietnamese

Vietnamese is a syllable-based tonal language where the tone used in syllable pronunciation carries important information about the meaning. In this paper, we investigate several approaches how to incorporate the tone into an acoustic model. We propose 3 basic strategies: a) a phoneme-based, b) a vowel-based, and c) a rhyme-based one. Each can be modified so that we obtain 15 different schemes that are described and compared in experiments performed within the framework of large-vocabulary continuous speech recognition of Vietnamese. We show that the phoneme-based context dependent model performs best, particularly when information about the tone is linked to the syllable end. On the test set, made of 85 minutes of mostly broadcast speech, we achieve 74% syllable accuracy rate. The accuracy is further improved to 78% when the pronunciation lexicon and the language model takes into account also 40,000 most frequent syllable pairs.