Multi-user real-time speech recognition with a GPU

We have developed a multi-user large vocabulary speech recognition system employing a fully composed one-level weighted finite state transducer (WFST) based network on a Graphics Processing Unit (GPU). This system improves the overall throughput and latency of speech recognition engine which processes multiple users´ utterances at the same time with efficient scheduling, parameter sharing, and communication overhead reduction techniques. We conduct both batch speech simulation and trace driven online simulation to access the performance of the developed system. Traces are generated based on a queueing model.