Title :
Understanding the dropout strategy and analyzing its effectiveness on LVCSR
Author :
Jie Li ; Xiaorui Wang ; Bo Xu
Author_Institution :
Interactive Digital Media Technol. Res. Center, Inst. of Autom., Beijing, China
Abstract :
The work by Hinton et al shows that the dropout strategy can greatly improve the performance of neural networks as well as reducing the influence of over-fitting. Nevertheless, there is still not a more detailed study on this strategy. In addition, the effectiveness of dropout on the task of LVCSR has not been analyzed. In this paper, we attempt to make a further discussion on the dropout strategy. The impacts on performance of different dropout probabilities for phone recognition task are experimented on TIMIT. To get an in-depth understanding of dropout, experiments of dropout testing are designed from the perspective of model averaging. The effectiveness of dropout is analyzed on a LVCSR task. Results show that the method of dropout fine-tuning combined with standard back-propagation gives significant performance improvements.
Keywords :
backpropagation; neural nets; probability; speech recognition; LVCSR; TIMIT; dropout fine-tuning method; dropout probabilities; dropout strategy; dropout testing; large vocabulary continuous speech recognition; model averaging; neural network performance improvement; over-fitting influence reduction; phone recognition task; standard backpropagation; Accuracy; Hidden Markov models; Neural networks; Standards; Testing; Training; Vectors; LVCSR; deep neural networks; dropout;
Conference_Titel :
Acoustics, Speech and Signal Processing (ICASSP), 2013 IEEE International Conference on
Conference_Location :
Vancouver, BC
DOI :
10.1109/ICASSP.2013.6639144