Nonnative Speech Recognition Based on State-Level Bilingual Model Modification

The performance of automatic speech recognition decreases drastically for nonnative speakers, especially those who are just beginning to learn foreign language or who have heavy accents. This paper presents a novel bilingual model modification approach to improve nonnative speech recognition via considering these great variations of accented pronunciations. Each state of baseline nonnative acoustic models is modified with several candidate states from auxiliary acoustic models, which are trained by speakers´ mother language. State mapping criterion and n-best candidates are investigated based on a grammar-constrained speech recognition system. Using the state-level bilingual model modification approach, compared to the nonnative acoustic models which have already been well trained by adaptation technique MAP, a relative reduction of 11.7% in phrase error rate (RPhrER) was further achieved.