Determining Degree of Relevance of Reviews Using a Graph-Based Text Representation

Reviews are text-based feedback provided by reviewers to authors. The quality of a review can be determined by identifying how relevant it is to the work that the review was written for as well as its similarity to existing well-written and coherent reviews. Relevance between two pieces of text can be determined by identifying semantic and syntactic similarities between them. In this paper, we make use of string-based metrics that incorporate concepts of paraphrasing and plagiarism to determine matching between texts. We use a graph-based text representation technique. We use the k-nearest neighbor classification algorithm to build a supervised model and classify text as LOW, MEDIUM or HIGH based on values of the metrics. We evaluate our approach on three data sets from student assignments and show that our model achieves an average accuracy of 63%.