Virtually Unplugged: Rich Data Capture to Evaluate CS Pedagogy in 3D Virtual Worlds

Being able to teach students about CS concepts effectively is a growing priority in primary schools. Teachers need to develop not just their content knowledge but also best- practice pedagogical approaches. Direct observation is one way of evaluating new teaching methods, but research with video/screen capture is typically time-consuming and difficult to get quick insights from. In this study we explore using 3D virtual worlds to quickly collect richer information from students working on an educational activity in a simulated 3D environment such as Second Life. We report on a case study exploring the insights that could be gleaned from this data collection, and compare how students learned in a 3D virtual world with simpler 2D map-based versions of a CS Unplugged problem-solving activity. The activity chosen was initially designed for physical use, and did not intrinsically favour either computer-based environment. Students using the 2D condition found a non-optimal solution faster, but the amount of extra time taken to find the optimal solution was similar to the 3D condition, and students had similar post-test scores in both conditions. Students using the 3D condition appeared to choose their actions more carefully, reflecting the higher physical load and slower response times of the virtual world environment. More students thought the 3D activity was fun and would choose to do again, and reported that it felt more like they were learning. While authoring tools are improving, virtual world content can still be challenging to develop and deploy. Care is needed to design engaging activities, and avoid creating environments that are immersive but a poor fit for learning.