The effects of fault counting methods on fault model quality

Over the past few years, we have been developing software fault predictors based on a system´s measured structural evolution. We have previously shown there is a significant linear relationship between code chum, a set of synthesized metrics, and the rate at which faults are inserted into the system in terms of number of faults per unit change in code chum. A limiting factor in this and other investigations of a similar nature has been the absence of a quantitative, consistent, and repeatable definition of what constitutes a fault. The rules for fault definition were not sufficiently rigorous to provide unambiguous, repeatable fault counts. Within the framework of a space mission software development effort at the Jet Propulsion Laboratory (JPL) we have developed a standard for the precise enumeration of faults. This new standard permits software faults to be measured directly from configuration control documents. Our results indicate that reasonable predictors of the number of faults inserted into a software system can be developed from measures of the system´s structural evolution. We compared the new method of counting faults with two existing techniques to determine whether the fault counting technique has an effect on the quality of the fault models constructed from those counts. The new fault definition provides higher quality fault models than those obtained using the other definitions of fault