Empirical Effectiveness Evaluation of Spectra-Based Fault Localization on Automated Program Repair

Researchers have proposed many spectra-based fault localization (SBFL) techniques in the past decades. Existing studies evaluate the effectiveness of these techniques from the viewpoint of developers, and have drawn some important conclusions through either empirical study or theoretical analysis. In this paper, we present the first study on the effectiveness of SBFL techniques from the viewpoint of fully automated debugging including the program repair of automation, for which the activity of automated fault localization is necessary. We assess the accuracy of fault localization according to the repair effectiveness in the automated repair process guided by the localization technique. Our experiment on 14 popular SBFL techniques with 11 subject programs shipping with real-life field failures presents the evidence that some conclusions drawn in prior studies do not hold in our experiment. Based on experimental results, we suggest that Jaccard should be used with high priority before some more effective SBFL techniques specially proposed for automated program repair occur in the future.