Learning Methods for Rating the Difficulty of Reading Comprehension Questions

This work deals with an Intelligent Tutoring System (ITS) for reading comprehension. Such a system could promote reading comprehension skills. An important step towards building a full ITS for reading comprehension is to build an automated ranking system that will assign a hardness level to questions used by the ITS. This is the main concern of this work. For this purpose we, first, had to define the set of criteria that determines the rate of difficulty of a question. Second, we prepared a bank of questions that were rated by a panel of experts using the set of criteria defined above. Third, we developed an automated rating software based on the criteria defined above. In particular, we considered and compared different machine learning techniques for the ranking system of the third part of the process: Artificial Neural Network (ANN), Support Vector Machine (SVM), decision tree and naïve Bayesian network. The definition of the criteria set for rating a question´s difficulty, and the development of an automated software for rating a questions´ difficulty, contribute to a tremendous advancement in the ITS domain for reading comprehension by providing a uniform, objective and automated system for determining a question´s difficulty.