Beyond multiple choice exams: Using computerized lexical analysis to understand students´ conceptual reasoning in STEM disciplines

Constructed response questions - in which students must use their own language in order to explain a phenomenon - create more meaningful opportunities for instructors to identify their students´ learning obstacles than multiple choice questions. However, the realities of typical large-enrollment undergraduate classes restrict the options faculty have for moving towards more learner-focused instruction. We are exploring the use of computerized lexical analysis of students´ writing in large enrollment undergraduate biology and geology courses. We have created libraries that categorize student responses with > 90% accuracy. These categories can be used to predict expert ratings of student responses with accuracy approaching inter-rater reliability among expert raters. These techniques also provide insight into students´ use of analogical thinking, a fundamental part of scientific modeling. These techniques have potential for improving assessment practices across STEM disciplines.