Curriculum based discriminative language model training

Discriminative language modeling is a technique used for correcting automatic speech recognition errors, and can be handled as a classification or a ranking problem. The aim of curriculum learning is to train the model with examples or concepts of gradually increasing level of difficulty. In this work, we use the classification and ranking versions of the perceptron algorithm and investigate three different curriculum learning approaches based on selection, ordering and clustering of the training examples. The results show that curriculum learning can help increase the performance of a classifying perceptron system, and with the ranking perceptron, it is possible achieve similar system performance with a shorter training time.