Chinese-English bilingual speech recognition

Two methods of construct a Chinese-English bilingual phone inventory are proposed and investigated. Our research focuses on a robust, suitable and compact phone combination of the two utterly different languages. The first method is to combine Chinese phonemes and English phonemes together. It can provide the required consistency with the western languages. The second method is to combine Chinese INITIALs and FINALs (IFs) with English phonemes in the bilingual acoustic modeling. Experimental results show that the first method is more compact and flexible in acoustic modeling than the second method. But the performance decreases significantly about 1.9% and 3.8% in Chinese and English test respectively. On the contrary, the second method achieves higher word accuracy than the first. Its performance degrades only 0.3% and 2.2% for two languages, but with more parameters included in acoustic model. Some issues of building this bilingual speech recognizer are also addressed.