An empirical analysis of equivalence partitioning, boundary value analysis and random testing

An experiment comparing the effectiveness of equivalence partitioning (EP), boundary value analysis (BVA) and random testing was performed, based on an operational avionics system of approximately 20000 lines of Ada code. The paper introduces an experimental methodology that considers all possible input values that satisfy a test technique and all possible input values that would cause a module to fail (rather than arbitrarily chosen values from these sets) to determine absolute values for the effectiveness for each test technique. As expected, an implementation of BVA was found to be most effective, with neither EP nor random testing half as effective. The random testing results were surprising, requiring just 8 test cases per module to equal the effectiveness of EP, although somewhere in the region of 50000 random test cases were required to equal the effectiveness of BVA