On measurement of operational security [software reliability]

Ideally, a measure of the security of a system should capture quantitatively the intuitive notion of `the ability of the system to resist attack´. That is, it should be operational, reflecting the degree to which the system can be expected to remain free of security breaches under particular conditions of operation (including attack). Instead, current security levels at best merely reflect the extensiveness of safeguards introduced during the design and development of a system. Whilst we might expect a system developed to a higher level than another to exhibit `more secure behaviour´ in operation, this cannot be guaranteed; more particularly, we cannot infer what the actual security behaviour will be from knowledge of such a level. In the paper we discuss similarities between reliability and security with the intention of working towards measures of `operational security´ similar to those that we have for reliability of systems. Very informally, these measures could involve expressions such as the rate of occurrence of security breaches (cf. rate of occurrence of failures in reliability), or the probability that a specified `mission´ can be accomplished without a security breach (cf. reliability function). This new approach is based on the analogy between system failure and security breach, but it raises several issues which invite empirical investigation. We briefly describe a pilot experiment that we have conducted to judge the feasibility of collecting data to examine these issues