A quantitative method for performance analysis of an isolated word ASR system

This paper introduces a new measure of confusion between phones, based on isolated word recognition tests. This metric combines the advantages of previous measures, and excludes their disadvantages. It can be used for comparing the performance of two speech recognizers at phone level, providing a useful design tool. The main advantage is that tests are made on any set of recorded words, but measure of confusion is evaluated for a particular phone versus another one, and at the same time it is vocabulary independent. Note that manual phone segmentation is not needed. Furthermore, a suitable combination of several tests allows to obtain useful statistical paired tests. The advantages of this new method are illustrated on the basis of both artificial examples and comparisons between real ASR systems.