Determining how much software assurance is enough? A value-based approach

A classical problem facing many software projects is how to determine when to stop testing and release the product for use. On the one hand, we have found that risk analysis helps to address such "how much is enough?" questions, by balancing the risk exposure of doing too little with the risk exposure of doing too much. In some cases, it is difficult to quantify the relative probabilities and sizes of loss in order to provide practical approaches for determining a risk-balanced "sweet spot" operating point. However, we have found some particular project situations in which tradeoff analysis helps to address such questions. In this paper, we provide a quantitative approach based on the COCOMO II cost estimation model and the COQUALMO qualify estimation model. We also provide examples of its use under the differing value profiles characterizing early startups, routine business operations, and high-finance operations in marketplace competition situation. We also show how the model and approach can assess the relative payoff of value-based testing compared to value-neutral testing based on some empirical results. Furthermore, we propose a way to perform cost/schedule/reliability tradeoff analysis using COCOMO II to determine the appropriate software assurance level in order to finish the project on time or within budget.