Investigating the accuracy of defect estimation models for individuals and teams based on inspection data

Defect content estimation approaches, based on data from inspection, estimate the total number of defects in a document to evaluate the quality of the product and the development process. Objective estimation approaches require a high-quality measurement process, potentially suffer from overfitting, and may underestimate the number of defects for inspections that yield few data points. Reading techniques for inspection, which focus the attention of the inspectors on particular parts of the inspected document, may influence their subjective estimates. In this paper we consider approaches to aggregate subjective estimates of individual inspectors in a team to alleviate individual bias. We evaluate these approaches with data from an experiment in a university environment where 177 inspectors in 30 teams inspected a software requirements document. Main findings of the experiment were that reading techniques considerably influenced the accuracy of inspector estimates. Further team estimates improved both estimation accuracy and variation compared to individual estimates and one of the best empirically evaluated objective estimation approaches.