Test Suite Reduction by Combinatorial-Based Coverage of Event Sequences

Combinatorial-based criteria are useful in several studies for test suite generation, prioritization, and minimization. In this paper, we extend previous work by using combinatorial-based criteria for test suite reduction. We use criteria that are based on combinatorial coverage of events and consider the order in which events occur. A simple combinatorial-based criterion covers t-way events and does not differentiate between the order of events. The event pair (e₁, e₂) is counted the same as if it occurs in the order (e₂, e₁). We also use two criteria that consider the order in which events occur since different orderings of events may be valuable during testing. First, the consecutive sequence-based criterion counts all event sequences in different orders, but they must occur adjacent to each other. The sequence-based criterion counts pairs in all orders without the requirement that events must be adjacent. We evaluate the new criteria on three GUI applications. We use 2way inter-window coverage in our studies. All of the 2way combinatorial-based criteria are effective in reducing the test suites and maintaining close to 100% fault finding effectiveness. Our future work examines larger test suites, higher strength coverage, techniques to partition event data, and further empirical studies.