Readability index of essays as an alternative to the scoring procedure in L2 academic writing

Samples of participants’ writing were scored by two raters using TOEFL writing scoring rubric. The readability index of each text was calculated through the use of six readability formulae and graphs, i.e., Flesch-Kincaid index, Reading Ease index, FOG index, SMOG formula, Fry’s graph, and Dale-Chall readability index. The scores given to each essay were later compared to the obtained readability indices through the use of Spearman rho correlation coefficient formula. The correlation coefficients obtained ranged from .05 to .15, none of which significant. This indicates that readability index of a text and the writing assessment procedure through holistic rubrics are dealing with two different constructs and have very little in common. This also calls into question the reliability and validity of some computerized assessment programs such as PEG, LSA, or E-rater, which take into account factors very similar to those examined in readability formulae.