Examining the difficulty pathways of can-do statements from a localized version of the CEFR

The Japanese adaptation of the Common European Framework of Reference (CEFR-J) is a tailored version of the Common European Framework of Reference (CEFR), designed to better meet the needs of Japanese learners of English. The CEFR-J, like the CEFR, uses illustrative descriptors known as can-do statements, that describe achievement goals for five skills (listening, reading, spoken production, spoken interaction and writing) across twelve levels instead of the CEFR’s original six. The goal of the present analysis is to provide validity evidence in support of the inherent difficulty hierarchy within the 5 A level sub-categories (A1.1, A1.2, A1.3, A2.1 and A2.2) in two ways: 1) by testing whether the difficulty of the can-do statements for each skill increases with the levels, and 2) by determining if there are significant differences in difficulty ratings between each level. It was found that for most skills, the rank ordering from difficulty ratings made by Japanese university students somewhat matched the level hierarchy of the CEFR-J but that significant differences between many adjacent levels were not found. The localization of a general framework for use by a specific population of users and the limitations related to using a system of can-dos that is derived from estimates of difficulty are discussed.