Speech act modeling and verification of spontaneous speech with disfluency in a spoken dialogue system

This work presents an approach to modeling speech acts and verifying spontaneous speech with disfluency in a spoken dialogue system. According to this approach, semantic information, syntactic structure and fragment class of an input utterance are statistically encapsulated in a proposed speech act hidden Markov model (SAHMM) to characterize the speech act. An interpolation mechanism is exploited to re-estimate the state transition probability in SAHMM, to deal with the problem of disfluency in a sparse training corpus. Finally, a Bayesian belief model (BBM), based on latent semantic analysis (LSA), is adopted to verify the potential speech acts and output the final speech act. Experiments were conducted to evaluate the proposed approach using a spoken dialogue system for providing air travel information. A testing database from 25 speakers, with 480 dialogues that include 3038 sentences, was established and used for evaluation. Experimental results show that the proposed approach identifies 95.3% of speech act at a rejection rate of 5%, and the semantic accuracy is 4.2% better than that obtained using a keyword-based system. The proposed strategy also effectively alleviates the disfluency problem in spontaneous speech.