On the Coverage of Program Code by Specification-Based Tests

Generating test cases from the specification can be done early in the software development life cycle. A test suite systematically generated from the software specification can be comprehensive in ensuring the intended functions to be exercised. Such a test suite is also instrumental in facilitating early discovery of software defects. On the other hand, an adequate level of code coverage is required to inspire confidence of the testing and quality assurance process. To examine the effectiveness of a specification-based test suite in covering program code, we conducted an exploratory case study which involved many different versions of programs that implemented the same specification. On the basis of the case study, this paper identifies the types of path in the programs, suggests enhancements to the existing test generation methods, and discusses several issues on how the code coverage can be improved.