Pitfalls of analyzing a cross-company dataset of software maintenance and support

It is important to establish a benchmark of work efficiency for software maintenance and support. For maintenance and support service providers, the benchmarking is the basis for improvement of their work. For the service purchasers, it is useful to check work efficiency of contracted service provider. To establish a benchmark of work efficiency for software development activities, a cross-company dataset is often used. Data points included in it are collected from various organizations. ISBSG (International Software Benchmarking Standards Group) builds the cross-company dataset of software maintenance and support. In the analysis, we found some pitfalls when one analyzes ISBSG software maintenance and support dataset. If the pitfalls are ignored, spurious relationships would be found. In this paper, we showed some pitfalls, and how to avoid it. It would be very useful for researchers, because ISBSG software maintenance and support dataset may be widely used in the near future.