An Account of Human and Automatic Methods for Evaluation of Machine Translation

This paper addresses the various methods of evaluation of machine translation systems’ output. In fact, machine translation (MT) is growing increasingly as people with different languages are being more connected, but such automatic translation systems have some problems with their outputs. Actually, for solving the dilemmas of translation systems’ outputs, any attempt to evaluate the results produced by these systems would facilitate insight into MT systems’ limitations and how they can be improved. Therefore, without employing a proper evaluation method, MT systems would not reveal their deficiencies and shortcomings. Evaluating the outputs of these translation software and evaluation methods has always been an important issue and an inseparable part of development in this field. Moreover, evaluation of MT systems has historically proven to be a difficult undertaking. However, a variety of methodologies have been proposed for evaluation of automatic translation systems which fall within two groups of human and automatic evaluation methods. Taking into consideration the requirements of and the undergoing process for evaluating the systems’ output quality, this paper focuses on the pros and cons of each of these methods with a focus on human evaluation methods. In particular, the paper discusses the traditional approaches of using human judges and the recent trend towards automatic methods.