Modeling long term variability information in a mixture stochastic trajectory framework

The problem of acoustic modeling for speech recognizers is addressed. We distinguish two types of speech variability, long term (speaker identity, stationary noise, channel distortion) and short term (phoneme class). Currently, most recognizers model the two variabilities without considering their specificities, which may result in flat distributions with limited discriminability. In our system, the long term variability (environment) is modeled by a mixture model, where each mixture is modeled by a mixture stochastic trajectory model (MSTM). We propose the environment dependent mixture stochastic trajectory model (ED-MSTM) to model a set of environments. The parameters of ED-MSTM are estimated using the maximum likelihood (ML) estimation criterion by the expectation-maximisation (EM) algorithm. Our model has been tested on a 1011 word vocabulary, multi-speaker continuous French recognition task with noisy speech. In the experiments, we assume that speakers can be grouped into a pre-determined number of classes and that the class label of a speaker is missing. The use of environmental modeling cut down the error rate produced by the multi-speaker system by about 15%, which is a statistically significant improvement. The idea of environment modeling is applicable to other acoustic modeling techniques such as hidden Markov models