Neural Network Language Models for Translation with Limited Data

In this paper we present how to estimate a continuous space language model with a neural network to be used in a statistical machine translation system. We report results for an Italian-English translation task obtained on a small corpus (about 150 K tokens), that can be considered a task with a lack of training data. Different word history length included in the connectionist language model (n-gram order) and distinct continuous space representation (i.e. words appearing in the training corpus more than k times) are considered in the study. The experimental results are evaluated by means of automatic evaluation metrics correlated with fluency and adequacy of the generated translations.