Vector quantization with memory and multi-labeling for isolated video-only automatic speech recognition

We describe a vector quantizer (VQ) with memory for automatic speech recognition (ASR) and compare the recognition performance results to those obtained with traditional memoryless VQ for ASR. Standard VQ for ASR quantizes the speech data independently of any past information. We introduce memory in a probabilistic framework for quantization state modeling. This is accomplished in the form of an ergodic hidden Markov model (HMM) in which the state occupied by the HMM represents the quantization label. We evaluate this approach in the context of video-only isolated digit ASR and implement both single stream (single labeling) and multi-stream (multi-labeling) systems. For single stream recognition, our approach increases the recognition rate from 62.67% to 66.95%. When using multi-labeling, our proposed vector quantizer with memory consistently outperforms the memoryless vector quantizer.