Towards an Information-Theoretic Approach for Measuring Intelligent False Alarm Reduction in Intrusion Detection

False alarms are a big challenge for intrusion detection systems (IDSs). A lot of approaches, especially machine learning based schemes, have been proposed to mitigate this issue by filtering out these false alarms. But a fundamental problem is how to objectively evaluate an algorithm in terms of its ability to correctly identify false alarms and true alarms. To improve the utilization of various machine learning algorithms, intelligent false alarm reduction has been proposed that aims to select and apply an appropriate algorithm in an adaptive way. Traditional metrics (e.g., true positive rate, false positive rate) are mainly used in the algorithm selection and evaluation, however, no single metric seems sufficient and objective enough to measure the capability of an algorithm in reducing false alarms. The lack of an objective and single metric makes it difficult to further fine-tune and evaluate the performance of algorithms in reducing IDS false alarms. In this paper, we begin by describing the relationship between the process of intrusion detection and the process of false alarm detection (reduction). Then we provide an information-theoretic analysis of intelligent false alarm reduction and propose an objective and single metric to evaluate different algorithms in identifying IDS false alarms. We further evaluate our metric under three scenarios by comparing it with several existing metrics.