Test suites for benchmarks of static analysis tools

This paper proposes test suites for the benchmarks of static analysis tools. In our test suites, a wide variety of common defects are intentionally implemented. In addition, we also propose several criteria for the evaluation of static analysis tools. By using them, we can compare the performance of static analysis tools in a quantitative manner. Our test suites are available in the public domain. Moreover, this paper provides the evaluation results of the top performance tools as references. We would like to anticipate the further improvement of static analysis tools leveraged by our test suites and reference data.