Investigation into the use of deep neural networks for LVCSR of Czech

This paper deals with utilization of deep neural networks (DNNs) for speech recognition. The main goal is to find out the best strategy for training and utilization of these models within an acoustic modeling module of a large vocabulary continuous speech recognition (LVCSR) system of Czech language. For this purpose, various DNNs are trained a) using several training strategies, b) with different inner structure and c) using various kinds of features. Experimental evaluation is then performed on a large dataset including broadcast recordings, recordings of lectures, dictates of judgments and set of nonlinearly distorted utterances. The resulting recipe for training of DNNs for our LVCSR system employs a) ReLU activation function with hidden layer width of 1024 neurons and b) filter-bank based features.