Comparing several coverage criteria for detecting faults in logical decisions

Many testing coverage criteria, including decision coverage and condition coverage, are well-known to be inadequate for software characterised by complex logical decisions, such as those in safety-critical software. In the past decade, more sophisticated testing criteria have been advocated. In particular, compliance of MC/DC has been mandated in the aviation industry for the approval of airborne software. On the other hand, the MUMCUT criterion has been proved to guarantee the detection of certain faults in logical decisions in irredundant disjunctive normal form. We analyse and empirically evaluate the ability of test sets satisfying these testing criteria in detecting faults in logical decisions. Our results show that MC/DC test sets are effective, but they may still miss some faults that can almost always be detected by test sets satisfying the MUMCUT criterion.