N-Best Rescoring for Speech Recognition using Penalized Logistic Regression Machines with Garbage Class

State-of-the-art pattern recognition approaches like neural networks or kernel methods have only had limited success in speech recognition. The difficulties often encountered include the varying lengths of speech signals as well as how to deal with sequences of labels (e.g., digit strings) and unknown segmentation. In this paper we present a combined hidden Markov model (HMM) and penalized logistic regression machine (PLRM) approach to continuous speech recognition that can cope with both of these difficulties. The key ingredients of our approach are N-best rescoring and PLRM with garbage class. Experiments on the Aurora2 connected digits database show significant increase in recognition accuracy relative to a purely HMM-based system.