Generation of Phonetic Units for Mixed-Language Speech Recognition Based on Acoustic and Contextual Analysis

This work presents a novel approach to generating phonetic units in order to recognize mixed-language or multilingual speech. Acoustic and contextual analysis is performed to characterize multilingual phonetic units for phone set creation. Acoustic likelihood is utilized for similarity estimation of phone models. The hyperspace analog to language (HAL) model is adopted for contextual modeling and contextual similarity estimation. A confusion matrix combining acoustic and contextual similarities between every two phonetic units is built for phonetic unit clustering. Multidimensional scaling (MDS) method is applied to the confusion matrix for reducing dimensionality. Experimental results indicate that the created phonetic set provides a compact and robust set that considers acoustic and contextual information for mixed-language or multilingual speech recognition.