The repeatability of code defect classifications

Counts of defects found during the various defect defection activities in software projects and their classification provide a basis for product quality evaluation and process improvement. However, since defect classifications are subjective, it is necessary to ensure that they are repeatable (i.e., that the classification is not dependent on the individual). We evaluate a slight adaptation of a commonly used defect classification scheme that has been applied in IBM´s Orthogonal Defect Classification work, and in the SEI´s Personal Software Process. The evaluation utilizes the Kappa statistic. We use defect data from code inspections conducted during a development project. Our results indicate that the classification scheme is in general repeatable. We further evaluate classes of defects to find out if confusion between some categories is more common, and suggest a potential improvement to the scheme