Colored Petri nets as the enabling technology in intrusion detection systems

Behavior based intrusion detection technologies are increasingly popular. Traditionally behavior patterns are expressed as specific signatures defined in the system call domain. This approach has various drawbacks and is vulnerable to possible obfuscations. The IDS approach discussed herein addresses process behavior in terms of functionalities, i.e. particular process objectives. The functionalities are formalized in the form that is independent of their specific realizations and is obfuscation resistant. The malware is detected by particular sets of functionalities exposed by programs during their execution. The approach implies the selection of common malicious functionalities, followed by formal description of these functionalities via specific system call combinations. In the detection domain, monitored system calls are combined into API functions utilizing Colored Petri nets (CPN). After that API functions are combined into malicious functionalities, indicative of malware attack, also using CPN. The advantages of CPN utilization for dynamic code analysis are described. By its nature the described approach is signature-based. The CPN technology is the backbone of the described approach: CPNs are used to define the functionalities of interests as behavior signatures, and at the same time serve as the mechanism for the signature detection. The paper describes a unique general-purpose software tool implementing CPN. It constitutes the enabling technology for the described IDS approach, and has many additional applications for modeling and monitoring complex hierarchical systems of discrete events.