Encoding Test Requirements as Constraints for Test Suite Minimization

Software (regression) testing is performed not only to detect errors as early as possible but also to guarantee that changes did not affect the system negatively. As test suites tend to grow over time, e.g., new test cases are added to test new features, (re-)executing the entire suite becomes prohibitive. We propose an approach, RZoltar, addressing this issue: it encodes the relation between a test case and its testing requirements (code statements in this paper) in a so-called coverage matrix, maps this matrix into a set of constraints, and computes a set of optimal solutions (maintaining the same coverage as the original suite) by leveraging a fast constraint solver. We show that RZoltar efficiently (0.68 seconds on average) finds a collection of test suites that significantly reduce the size of the original suite (61.12%), while greedy only finds one solution with a reduction of 56.58% in 6.92 seconds on average.