Hidden mode HMM using Bayesian network for modeling speaking rate fluctuation

One of the most important issues in spontaneous speech recognition is how to cope with the degradation of recognition accuracy due to speaking rate fluctuation within an utterance. This paper proposes an acoustic model for adjusting mixture weights and transition probabilities of the HMM for each frame according to the local speaking rate. The proposed model is implemented along with variants and conventional models using the Bayesian network framework. The proposed model has a hidden variable representing variation of the "mode" of the speaking rate and its value controls the parameters of the underlying HMM. Model training and maximum probability assignment of the variables are conducted using the EM/GEM and inference algorithms for Bayesian networks. Utterances from meetings and lectures are used for evaluation where Bayesian network-based acoustic models are used to rescore the utterance hypotheses obtained from a first-pass N-best list. In the experiments, the proposed model shows consistently higher performance than conventional models.