An industrial case study of the effectiveness of test generators

Automatic test generators pursue some type of systematic coverage of the program code or heuristic sampling of the program inputs. Test generators are effective after the assumption, often (enthusiastically) embraced by researchers, that the generated test cases produce informative data for domain experts, e.g., pinpoint important bugs. This paper investigates the validity of such assumption through a case study of using test generators on industrial software with nontrivial domain-specific peculiarities. Our results properly enhance the available body of knowledge on the strengths and weaknesses of test generators.