Recomputing Coverage Information to Assist Regression Testing

This paper presents a technique that leverages an existing regression test selection algorithm to compute accurate, updated coverage data on a version of the software, P_i+1, without rerunning any test cases that do not execute the changes from the previous version of the software, P_i to P_i+1. The technique also reduces the cost of running those test cases that are selected by the regression test selection algorithm by performing a selective instrumentation that reduces the number of probes required to monitor the coverage data. Users of our technique can avoid the expense of rerunning the entire test suite on P_i+1 or the inaccuracy produced by previous approaches that estimate coverage data for P_i+1 or that reuse outdated coverage data from Pi. This paper also presents a tool, RECOVER, that implements our technique, along with a set of empirical studies on a set of subjects that includes several industrial programs, versions, and test cases. The studies show the inaccuracies that can exist when an application-regression test selection-uses estimated or outdated coverage data. The studies also show that the overhead incurred by selective instrumentation used in our technique is negligible and overall our technique provides savings over earlier techniques.