Evaluating the performance of speech recognisers at the acoustic-phonetic level

At some stage of the recognition process, a choice has to be made between lexical items. This choice is the most difficult if the items form a minimal series (they differ only by one phoneme). A selection of such minimal series has been used to test a number of commercially available word recognition systems (CNET ´Dynamo´, INTERSTATE ´VRM´, LIMSI-VECSYS ´Primo´, THRESHOLD ´T 600´) and several speech recognizers developped in France. Recognition scores beeing highly dependant on the quality of the acoustic samples used for training and testing, performance of a system is expressed as the noise level necessary to obtain identical human recognition scores on the same test material. Confusion matrices can be used by the experimenter to correct defficiencies of the algorithms and techniques used, or by the user to select an application vocabulary.