Towards Better Fault Localization: A Crosstab-Based Statistical Approach

It is becoming prohibitively expensive and time consuming, as well as tedious and error-prone, to perform debugging manually. Among the debugging activities, fault localization has been one of the most expensive, and therefore, a large number of fault-localization techniques have been proposed over the recent years. This paper presents a crosstab-based statistical technique that makes use of the coverage information of each executable statement and the execution result (success or failure) with respect to each test case to localize faults in an effective and efficient manner. A crosstab is constructed for each executable statement, and a statistic is computed to determine the suspiciousness of the corresponding statement. Statements with a higher suspiciousness are more likely to contain bugs and should be examined before those with a lower suspiciousness. Case studies are performed on both small- (the Siemens and Unix suites) and large-sized programs (space, grep, gzip, and make), and results suggest that the crosstab-based technique (CBT) is more effective (in terms of a smaller percentage of executable statements that have to be examined until the first statement containing the fault is reached) than other techniques, such as Tarantula. Further studies using the Siemens suite reveal that the proposed technique is also more effective at locating faults than other statistically oriented techniques, such as SOBER and Liblit05. Additional experiments evaluate the CBT from other perspectives, such as its efficiency in terms of time taken, its applicability to object-oriented languages (on a very large Java program: Ant), and its sensitivity to test suite size, and demonstrate its superior performance.