Software assessment: ten years on

Consideration is given to how software assessment and certification has developed since the author first became involved in the early 1980s. The author concentrates on safety-related software, although many of the conclusions are applicable to mission-critical and security-critical systems. In 1983, software assessment was bedevilled by a lack of both theory and experience. There were no generally applicable standards. In the absence of a satisfactory theory for software reliability, assessment was based on an `argument of excellence´, which sought to show that the safety integrity of the software components was high enough not to compromise the safety integrity of the hardware. The argument of excellence depended on a three-legged assessment approach: quality audit; analysis; and testing. In 1993, matters have improved considerably, although several problems still remain. There are a number of standards that can be used for the basis of assessment, including IEC SC65A WG9, IDS 00-55, DO178B, DIN VDE 0801 and various standards and drafts to support the Machinery Directive. The three-legged approach is still applicable, but the analysis leg is supported by progress in techniques and tools for static analysis and formal methods, including powerful theorem provers from North America and the SPARK Examiner. Static analysis as part of assessment is happening on a heroic scale. Software reliability theory has also advanced