Creating a self-rating instrument for second language writing: from idea to implementation

This article reports on the development of a self-rating instrument for writing. The instrument engages learners in responding to a writing task and assessing their own proficiency against a set of benchmarks. The technical and conceptual framework for the instrument is the DIALANG project, a diagnostic language assessment system for 14 languages on the internet. The report comprises a description of the self-rating procedure, an account of instrument development, a report on a usability study with six learners of Finnish as a second language, and our test-developer analysis of the process and products of the self-rating instrument. The data consists of records of task and benchmark development, participant background data, the texts that the learners wrote, video recordings of their screen actions during the self-rating process, immediate retrospective interviews with the learners, and teacher ratings of the learner texts. The discussion concentrates on the concepts that underlie the self-rating process and the potential offered by this approach for the learning, teaching, and assessment of second language writing.