Performance metric alerting: a new design approach for complex alerting problems

Alerting systems and related decision-making automation are widely used to enhance the safety and capability of controlled processes across many applications. Traditional alerting systems use physical metrics such as temperature, distance between vehicles, or time-to-impact as bases for making alerting decisions. Threshold values on these metrics are typically derived using an iterative process to ensure the achievement of desired performance goals, defined by higher-level metrics such as false alarm, safety, or success probabilities. We generalize this problem and develop two state spaces: one representing physical metrics and one representing performance metrics. A traditional alerting system operates completely within the physical space, using decision thresholds that have been developed off-line during the design process by examining how the physical threshold translates across to the performance state space. The physical metrics thus act as an indirect means to control the performance of the system. We propose an alternate approach that enables the system to operate online in the performance space. Alerting decisions are based directly on the computed values of metrics such as false alarm probability rather than on surrogate physical metrics. These two design approaches are then contrasted in case studies of recently-developed alerting systems