Empirical and statistical analysis of risk analysis-driven techniques for threat management

One of the challenges of secure software construction (and maintenance) is to get control over the multitude of threats in order to focus mitigation efforts on the most relevant ones. Risk analysis is one class of techniques for achieving threat reduction, but few studies are available that evaluate the quality of these techniques. In this paper, a selected set of risk analysis techniques have been evaluated and compared based on a realistic case study. The foundations for this analysis were threefold: we defined a set of high-level criteria, we compared the results of the different methods and we used statistical analysis techniques for studying additional characteristics. This analysis was performed on an independently developed case study of a significant size. For this experiment, the benefits of applying of these methods were limited for the categorization and the reduction of threats. Therefore, we also suggest ways to improve or complement these methods