Characterizing effects of adaptivity within an intelligent tutoring system

This study aims to characterize the effects of adaptivity within an intelligent tutoring system (ITS). An ITS authoring tool was used to create adaptive and non-adaptive (sequential) versions of lessons on programmable logic controller timer instructions. In the adaptive versions, the content and number of practice questions asked varied based on student performance. The sequential versions were identical to the adaptive versions except that they asked a practice questions from a predetermined list. Students in both groups showed statistically significant improvement in post-test performance, and there was no difference between groups in the amount of improvement. Students in the sequential group answered practice questions more accurately than students in the adaptive group. However, students in the adaptive group required significantly less time to complete the lessons. Correlation data suggest that the adaptive lessons were more efficient for knowledgeable students. This research suggests that adaptive approaches can make learning more efficient, but are not necessarily more effective than pedagogically sound non-adaptive approaches. It also demonstrates effective data collection and analysis techniques for studying ITS features such as adaptivity. Use of these techniques can facilitate the design of instructionally effective ITSs and inform design of other tools for ITS research and evaluation.