Leveraging a Constraint Solver for Minimizing Test Suites

Software (regression) testing is performed to detect errors as early as possible and guarantee that changes did not affect the system negatively. As test suites tend to grow over time, (re-)executing the entire suite becomes prohibitive. We propose an approach, RZoltar, addressing this issue: it encodes the relation between a test case and its testing requirements (code statements in this paper) in a so-called coverage matrix, maps this matrix into a set of constraints, and computes a collection of optimal minimal sets (maintaining the same coverage as the original suite) by leveraging a fast constraint solver. We show that RZoltar efficiently (0.95 seconds on average) finds a collection of test suites that significantly reduce the size (64.88% on average) maintaining the same fault detection (as initial test suite), while the well-known greedy approach needs 11.23 seconds on average to find just one solution.