Towards a Benchmark and Automatic Calibration for IR-Based Concept Location

There has been a great deal of research into the use of Information Retrieval (IR)-based techniques to support concept location in source code. Much of this research has been focused on determining how to use various IR techniques to support concept location. Very little attention has been given to the effect of different configurations of corpus building and indexing on query results. In this paper, we propose a tool designed to support large-scale studies of IR techniques in varying configurations of parameters with the intention of automatically calibrating these parameters. We also discuss preliminary efforts to create the benchmark data such studies require.