Efficiently Running Test Suites Using Abstract Undo Operations

The last decade has seen many advances in test input generation, specifically using systematic approaches that can enumerate many tests. While such approaches have enhanced our ability to find bugs in programs, running large numbers of tests remains a time consuming and expensive task, especially for tests that execute operations on external resources, such as a file system or a network. This paper presents a novel technique for optimizing execution of suites of tests, where several tests in a suite may contain common initial execution -- a property often exhibited by systematically generated suites, e.g., those for bounded exhaustive testing. Our insight is that we can cluster execution of such tests by defining abstract-level undo operations, which allow a common execution segment to be performed once, and its result to be shared across the tests, which then perform the rest of their operations. We present our framework for clustered execution of test suites, and evaluate it using three case-studies, which show our technique enables significant performance speed-up over traditional test execution.