Testability, failure rates, detectability, trustability and reliability

Discusses the relationship between several statistical measures of program dependability, including failure rates and testability. This is done by describing these concepts within the framework of a confidence-based measure called trustability. Suppose that M is a testing method, F is a class of faults and P is a class of programs. Suppose that the probability of a fault from F causing a failure is at least D when a program p∈P is tested according to M, if in fact p contains a fault of type F. Then D is called the detectability of M with respect to F and P. If we test a program using a method with detectability D, and see no faults, then we can conclude with risk at most 1-D that the program has no faults, i.e. we can have confidence at least C=D that the program is fault-free for the associated fault class F. If we have confidence at least C that a program has no faults, then we say that the program has trustability C with respect to F. More refined measures of trustability can be defined which also take fault class frequencies into account. Testability is defined to be the probability of finding a fault in a program p, if p contains a fault. The probability that a program will fail when it is tested over its operational distribution is called its failure rate. Trustability is confidence in the absence of faults and reliability is the probability of a program operating without failure. Trustability and reliability coincide if the class of faults for which we have a certain level of trustability is the class of common case faults