Peer-Supervised Learning with Built-In Quality Control Based on Multiple-Choice Questions: A Case Study

Rating multiple choice questions (MCQ) created by peers has been touted as a good approach to peer assessment. The main challenge in this setting is to ensure the quality of peer assessment. Existing approaches rely on the assumption that students intrinsically create high-quality ratings. We propose an incentive mechanism to increase the quality of ratings. To evaluate our approach, we have conducted a case study with 242 students and 17 experts. Our results show that peer ratings are a good predictor of expert ratings. Furthermore, we develop a model that reliably measures the performance of students, but does not require expert ratings.