Specification-Based Testing of Intrusion Detection Engines Using Logical Expression Testing Criteria

An Intrusion Detection System (IDS) protects computer networks against attacks and intrusions. One class of IDS is called signature-based network IDSs as they monitor network traffic, looking for evidence of malicious behaviour as specified in attack descriptions (referred to as signatures). Many studies report that IDSs have problems accurately identifying attacks. Therefore, it is important to precisely understand under which conditions IDSs accurately identify attacks or fail to do so. However, no systematic approach has so far been defined and used to study this problem. Recognizing that signatures in essence provide the specification of an IDS engine, studying the accuracy of an IDS engine becomes a black-box testing problem. We therefore precisely and systematically evaluate which mature testing techniques can be used (and adapted) to derive tests from IDS signatures. We experiment with those criteria on one widely used and maintained IDS and show that our approach is effective at systematically revealing problems in this IDS engine (e.g., problems that prevent the detection of attacks).