Non-native teachers’ rating criteria for L2 speaking: Does a rater training program make a difference?

Inconsistent rating seems to emanate largely from the application of different rating criteria. It follows that rater training programs may bring about higher rating consistency. This study aimed to explore non-native EFL teachers’ rating criteria for L2 learners’ speaking performance and to measure the impact of a rater training program on raters’ rating criteria. As many as 28 EFL teachers rated 10 monologs both before and subsequent to a rater training program and specified the criteria they applied in their ratings. The findings show they specified 10 common rating criteria, ranging from fluency to communicative effectiveness. However, they reconsidered the significance of a few criteria after the program. While there was a sharp decline in the significance given to the rate of speech and affective variables, the training program led to the rising importance of fluency, comprehension, and organization. The results reveal that the traditional skills-and-components-based perspective on language proficiency makes teachers lose sight of macro-level, higher-order components like fluency and organization. To conclude, the effective rating of language skills needs to be embedded in teacher education programs