Empirical evaluation of existing algorithms of spectrum based fault localization

Fault localization is an essential step for debugging, even though it is still tedious and time-consuming activity. Many researchers tried to find a good way for it for decades. Many solutions proposed by them have different performance, such as correctness, code coverage, and etc. However there are few attempts to compare those solutions in an objective way. It makes software testers confuse to choose a solution for their purposes. In this paper, we perform empirical evaluations for existing 32 algorithms that are prominent in the domain of fault localization. Through the evaluation we analyze them with some properties such as correctness and categorize them with a clustering method. All the experimental effort has been performed with a tool, named SKKU-FL, which has been developed by us. The ultimate goal of this paper is to provide an insight to classify a large number of existing solutions and help testers to select the most appropriate solution for their aims.