Using the cognitive walkthrough to improve the design of a visual programming experiment

Visual programming languages aim to promote usability, but are rarely examined for it. One reason is the difficulty of designing successful experimental evaluations. We propose the cognitive walkthrough as an aid to improve experimental designs. This is a novel application of an HCI-derived technique designed for evaluating interfaces rather than experiments. The technique focuses on the potential difficulties of novice users and is therefore particularly suited for evaluating the programming situation, which is knowledge-based and non-routine. We describe an empirical study performed without benefit of a walkthrough and show how the study was improved by a series of walkthroughs. We found the method to be quick to use, effective at improving the experimental design, and usable by non-specialists