Quantitative evaluation and operative usage of interactive systems

Probabilistic safety assessment of computer based systems, and, more generally, all the quantitative evaluations of system dependability, require an estimate of the possible operative usage of the system under evaluation. The complexity of the real operative usage of computer based systems necessitates several approximations and assumptions during the estimation. These can be particularly significant for systems having a large number of interactions with human agents, such as process control and decision support systems. This paper reports the example of a computer based system supporting train drivers in respecting line signals and speed restrictions. The originally conducted safety assessment missed to consider a possible operative usage of this system, which was adopted by the train drivers. The system failed in preventing the passage of red signals by train drivers several times, and one of the resulting incidents is analysed in the paper to show the difference between the estimated and the real operative usage. The paper concludes by emphasising the hidden dangers of quantification in safety assessment, especially when based on assumptions concerning the possible behaviour of the human agents interacting with the computerised part of the system under evaluation.