On the Verification and Validation of Signature-Based, Network Intrusion Detection Systems

An Intrusion Detection System (IDS) protects computer networks against attacks and intrusions in combination with firewalls and anti-virus systems. One class of IDS is called signature-based network IDSs as they monitor network traffic, looking for evidence of malicious behaviour as specified in attack descriptions (referred to as signatures). It is common knowledge in the research community that IDSs have problems accurately identifying attacks. In this paper we discuss this accuracy problem and decompose it into a detection problem and a confirmation problem. We then map the evaluation of this accuracy problem to the traditional software verification and validation problem, which allows us to analyze the techniques academics have been using to evaluate their IDS technologies. As a result, we are able to identify areas where research is needed to improve the assessment of the IDS accuracy problem through verification and validation techniques.