Structuring Safety Cases for Autonomous Systems

Def Stan 00-56 requires a safety case to be built before an autonomous system can be certified, but there is no current guidance on how such a case should be structured. The authors have reviewed several plausible approaches to structuring a safety case, including arguing human equivalence, deriving necessary capabilities from a level of autonomy scheme, and by deriving an explicit rationale for the unmanned systems safety guide recently published by the US Department of Defense. From this, we have produced an initial recommended approach. The process of deriving it has revealed that much of the published advice on autonomous system safety is either of very low value or potentially dangerous.