H- and C-level WFST-based large vocabulary continuous speech recognition on Graphics Processing Units

We have implemented 20,000-word large vocabulary continuous speech recognition (LVCSR) systems employing Hand C-level weighted finite state transducer (WFST) based networks on Graphics Processing Units (GPUs). Both the emission probability computation and the Viterbi beam search are implemented on the GPU in a data-parallel manner to minimize the extra data transfer time between the host CPU and the GPU. This study utilizes word-length optimization techniques to reduce the synchronization overhead in the Viterbi beam search. We achieve 18.6% to 21.9% of speed up by using an efficient data packing method with less than 0.2% accuracy degradation. Furthermore, we explore different levels of abstraction in recognition network generation to reduce the number of synchronization operations as well as to minimize the memory usage. The experimental results show that the implemented systems on the GPU perform speech recognition 4.07 to 4.55 times faster than highly optimized sequential implementations on a CPU.