An evaluation of computerised essay marking for national curriculum assessment in the UK for 11-year-olds

This paper reports a comparison of human and computer marking of approximately 600 essays produced by 11-year-olds in the UK. Each essay script was scored by three human markers. Scripts were also scored by the e-rater program. Therewas a good agreement between human and machine marking. Scripts with highly discrepant scores were flagged and assessed blind by expert markers for characteristics considered likely to produce human–machine discrepancies. As hypothesised, essays marked higher by humans exhibited more abstract qualities such as interest and relevance, while there was little, if any, difference on more mechanical factors such as paragraph demarcation.