Experimental evaluation of the variation in effectiveness for DC, FPC and MC/DC test criteria

Given a test criterion, the number of test-sets satisfying the criterion may be very large, with varying fault detection effectiveness. This paper presents an experimental evaluation of the variation in fault detection effectiveness of all the test-sets for a given control-flow test criterion and a Boolean specification. The exhaustive experimental approach complements the earlier empirical studies that adopted analysis of some test-sets using random selection techniques. Three industrially used control-flow testing criteria, decision coverage (DC), full predicate coverage (FPC) and modified condition/decision coverage (MC/DC) have been analysed against four types of faults. The Boolean specifications used were taken from a past research paper and also generated randomly. To ensure that it is the test-set size, a variation of DC, decision coverage/random (DC/R), has also been considered against FPC and MC/DC criteria. In addition, a further analysis of variation in average effectiveness with respect to number of conditions in the decision has been done. The empirical results show that the MC/DC criterion is more reliable and stable in comparison to DC, DC/R and FPC.