Estimating the number of undetected errors: Bayesian model selection

Sometimes complex software systems fail because of faults introduced in the requirements and design stages of the development process. To improve the reliability of a software, several reviewers inspect documents related to requirements and design. Some faults are detected in this process but often a few remain undetected until the software is developed. Ebrahimi (1997) developed frequentist methods to estimate the number of faults which are nor discovered. Later, Basu and Ebrahimi and Basu developed different Bayesian models. These different Bayesian models yield different estimates of the number of undetected errors. It was further found that changes in the prior parameters often result in measurable changes in the estimates of undetected errors. The authors address the issue of model selection among these different models. They advocate two model selection methods. The first method uses marginal likelihood and Bayes factor whereas the second method is based on cross-validated likelihood. These methods are illustrated in the software review data of AT&T 5 ESS switches