A controlled experiment to assess the effectiveness of inspection meetings

Software inspection is one of the best practices for detecting and removing defects early in the software development process. In a software inspection, review is first performed individually and then by meeting as a team. In the last years, some empirical studies have shown that inspection meetings do not improve the effectiveness of the inspection process with respect to the number of true discovered defects. While group synergy allows inspectors to find some new defects, these meeting gains are offset by meeting losses, that is defects found by individuals but not reported as a team. We present a controlled experiment with more than one hundred undergraduate students who inspected software requirements documents as part of a university course. We compare the performance of nominal and real teams, and also investigate the reasons for meeting losses. Results show that nominal teams outperformed real teams, there were more meeting losses than meeting gains, and that most of the losses were defects found by only one individual in the inspection team