A Novel Evaluation Method for Defect Prediction in Software Systems

In this paper, we propose a novel evaluation method for defect prediction in object-oriented software systems. For each metric to evaluate, we start by applying it to the dependency graph extracted from the target software system, and obtain a list of classes ordered by their predicted degree of defect under that metric. By utilizing the actual defect data mined from the subversion database, we evaluate the quality of each metric through means of a weighted reciprocal ranking mechanism. Our method can tell not only the overall quality of each evaluated metric, but also the quality of the prediction result for each class, especially those costly ones. Evaluation results and analysis show the efficiency and rationality of our method.