How to Measure Software Reliability and How Not To

The paper criticises the underlying assumptions which have been made in much early modeling of computer software reliability. The following suggestions will improve modeling. 1) Do not apply hardware techniques to software without thinking carefully. Software differs from hardware in important respects; we ignore these at our peril. In particular-2) Do not use MTTF, MTBF for software, unless certain that they exist. Even then, remember that- 3) Distributions are always more informative than moments or parameters; so try to avoid commitment to a single measure of reliability. Anyway- 4) There are better measures than MTTF. Percentiles and failure rates are more intuitively appealing than means. S) Software reliability means operational reliability. Who cares how many bugs are in a program? We should be concerned with their effect on its operation. In fact- 6) Bug identification (and elimination) should be separated from reliability measurement, if only to ensure that the measurers do not have a vested interest in getting good results. 7) Use a Bayesian approach and do not be afraid to be subjective. All our statements will ultimately be about our beliefs in the quality of programs. 8) Do not stop at a reliability analysis; try to model life-time utility (or cost) of programs. 9) Now is the time to devote effort to structural models. 10) Structure should be of a kind appropriate to software, e.g. top-down, modular.