Towards the evaluation of software engineering standards

The paper presents the results of work undertaken during the first half of a project whose aim is to develop a measurement based approach for assessing software engineering standards. Using a simple model of a standard as a set of requirements the authors found that it is impossible to make an objective assessment of conformance to the majority of requirements. They make recommendations for improving standards. These include the restriction of individual requirements to a specific product, process or resource, with conformance able to be assessed objectively; the restriction of standards to cohesive collections of requirements; a clear statement of benefit. The approach to evaluation of a specific standard has two stages: firstly the rationalisation and decomposition into mini-standards and secondly the (ideal) measurement-based assessment. The initial stages of a major industrial case study being used to assess the approach are described. They show, by the example of the evaluation of a specific mini-standard, that data the company already collects, and which are similar to the type of data collected by other companies, can be used to form the basis of the assessment