Data Unpredictability in Software Defect-Fixing Effort Prediction

The prediction of software defect-fixing effort is important for strategic resource allocation and software quality management. Machine learning techniques have become very popular in addressing this problem and many related prediction models have been proposed. However, almost every model today faces a challenging issue of demonstrating satisfactory prediction accuracy and meaningful prediction results. In this paper, we investigate what makes high-precision prediction of defect-fixing effort so hard from the perspective of the characteristics of defect dataset. We develop a method using a metric to quantitatively analyze the unpredictability of a defect dataset and carry out case studies on two defect datasets. The results show that data unpredictability is a key factor for unsatisfactory prediction accuracy and our approach can explain why high-precision prediction for some defect datasets is hard to achieve inherently. We also provide some suggestions on how to collect highly predictable defect data.