Examining Factors Affecting Language Performance: A Comparison of Three Measurement Approaches

This article presents the findings of three approaches, namely, Classical Test Theory, Generalizability Theory (Brennan, 2001; Shavelson Webb, 1991) and Multi-facet Rasch Analysis (Linacre, 2004; McNamara, 1996) in examining the effects of different factors on language test performance. Through these approaches, the investigator sought to determine the extent test takers, raters, and test tasks contribute to the source of variance in performance on writing. Additionally, the investigator also examined whether raters’ severity and task difficulty affect the reliability and dependability of the observed language performance scores. The purpose of using these measurement approaches was to determine whether findings could be integrated so as to strengthen validity arguments for test score dependability and generalizability.