Software release control using defect based quality estimation

We describe two case studies to investigate the application of a state variable model to control the system test phase of software products. The model consists of two components: a feedback control portion and a model parameter estimation portion. The focus in this study is on the assessment of the goodness of the estimates and predictions of the model parameters and their utility in the management of the system test phase. Two large network management applications developed and tested at Sun Microsystems served as the subjects in these studies. Unlike the release of products based on marketing or deadline pressure, estimates of the number of residual defects are used to control the quality of the product being released. The estimates of the number of defects in the application when the test phase began and at the current checkpoint are obtained. In addition a prediction is made regarding the reduction in the number of remaining defects over the remaining period. The estimates and predictions assist the management in planning the test phase and allow inferring the level of customer support needed subsequent to product release. The results of both case studies are satisfactory and, when viewed in light of other studies conducted at Sun Microsystems, show the applicability of the state variable model to the management of the software test process.