Study on the relevance of the warnings reported by Java bug-finding tools

Several bug-finding tools have been proposed to detect software defects by means of static analysis techniques. However, there is still no consensus on the effective role that such tools should play in software development. Particularly, there is still no concluding answer to the following question usually formulated by software developers and software quality managers: how relevant are the warnings reported by bug finding tools? The authors first report an in-depth study involving the application of two bug-finding tools (FindBugs and PMD) in five stable versions of the Eclipse platform. Next, in order to check whether the initial conclusions are supported by other systems, the authors describe an extended case study with 12 systems. In the end, it has been concluded that rates of relevance superior to 50% can be achieved when FindBugs is configured in a proper way. On the other hand, in the best scenario considered in the research, only 10% of the warnings reported by PMD have been classified as relevant.