Exploiting acoustic feature correlations by joint neural vector quantizer design in a discrete HMM system

In previous work about hybrid speech recognizers with discrete HMMs we have shown that VQs, that are trained according to a maximum mutual information (MMI) criterion, are well suited for ML estimated Bayes classifiers. This is only valid for single VQ systems. In this paper we extend the theory to speech recognizers with multiple VQs. This leads to a joint training criterion for arbitrary multiple neural VQs that considers the inter VQ correlation during parameter estimation. The idea of a gradient based joint training method is derived. Experimental results indicate that inter VQ correlations can cause some degradation of recognition performance. The joint multiple VQ training decorrelates the quantizer labels and improves system performance. In addition the new training criterion allows for a less careful way of splitting up the feature vector into multiple streams that do not have to be statistically independent. In particular the use of highly correlated features in conjunction with the novel training criterion in the experiments leads to important gains in recognition performance for the speaker independent resource management database and gives the lowest error rate of 5.0% we ever obtained in this framework