Confidence measures for detecting speech recognition errors

Errors of speech recognition systems occur due to a variety of reasons. It is desirable to have a confidence measure that gives an idea of the accuracy of the decoder output, so that appropriate remedial measures can be taken. In this paper, we compare two approaches to detect incorrect output of a speech recognition system. The first approach employs multiple decoders, and uses a voting method to surmise confidence in the accuracy of the speech recognition system. The second approach uses a single decoder, but judiciously combines information at the segmental as well as supra segmental level to derive a measure of confidence in the output of the decoder. A neural network is trained with three features based on phone duration and one feature based on acoustic score. The output of the neural network is used to estimate the confidence in the output of the decoder. The two approaches are compared for their efficacy in detecting utterances that do not contain a valid input according to the task grammar as well as wrongly recognized valid inputs. It was observed that the second method achieves much better rejection of invalid input utterances as compared to the multi-decoder method, despite decoding a test utterance just once.