On evaluation of interactive-predictive machine translation: Practical problems and suggestions

This paper first gives an overview on interactive-predictive machine translation systems. In such systems, the human translator translates the text while the machine provides completion to it. The user is given options to accept either the entire or a part of the suggested completion. Moreover, the user is also permitted to skip the suggestions and continue typing. Thereafter, we focus on TransType2 project, its architecture, and its evaluation methods. Furthermore we analyze the user log files of the last TransType2 Evaluation round. Eventually, according to the obtained analysis results, TransType2 project deliverables, and reports of human translators, we recommend several new features and adjustments to the current evaluation methodology of interactive-predictive machine translation systems. These new methods take advantage of the existing systems, add evaluation methods to them and modify them to result in qualified systems.