Dealing with acoustic mismatch for training multilingual subspace Gaussian mixture models for speech recognition

The subspace Gaussian mixture model (SGMM) has been recently proposed as an acoustic modeling technique suitable for configuring multilingual speech recognition systems. It is attractive for this purpose since its parametrization allows its “shared” model parameters to be trained with data from multiple languages [1]. In this work, we report on the results of an experimental study carried out with the goal of improving native Spanish language speech recognition performance using an existing telephone speech corpus of English spoken by speakers of Spanish origin. Compensation for sources of acoustic variability between Spanish and English language data sets was found to be important in obtaining good multilingual ASR performance. We conclude with a discussion about the notion of acoustic similarity between the state dependent parameters of the SGMM, and its possible use in effectively modelling pronunciation variation.