Defects in natural language requirement specifications at Mercedes-Benz: An investigation using a combination of legacy data and expert opinion

Natural language (NL) requirement specifications are widely used in industry, but ensuring high quality in these specifications is not easy. This work investigates in an empirical study the typical defect type distributions in current NL requirement specifications. For this study, more than 5,800 review-protocol-entries that originate from reviews of real automotive specifications according to a quality-model were categorized by us at MercedesBenz. As a result, we obtained (a) a typical defect type distribution in NL specifications in the automotive domain, (b) correlations of quality criteria to defect severity, (c) indicators on ease of handling quality criteria in the review-process and (d) information on time needed for defect correction with respect to quality criteria. To validate the findings from the data analysis, we additionally conducted 15 interviews with quality managers. The results confirm quantitatively that the most critical and important quality criteria in the investigated NL requirement specifications are consistency, completeness, and correctness.