Considering both intra-pattern and inter-pattern anomalies for intrusion detection

Various approaches have been proposed to discover patterns from system call trails of UNIX processes to better model application behavior. However, these techniques only consider the relationship between system calls (or system audit events). We first refine the definition of maximal patterns given in (Wespi et al., 2000) and provide a pattern extraction algorithm to identify such maximal patterns. We then add one additional dimension to the problem domain by also taking into consideration the overlap relationship between patterns. We argue that an execution path of an application is usually not an arbitrary combination of various patterns; but rather, they overlap each other in some specific order. Such overlap relationship characterizes the normal behavior of the application. Finally, a novel pattern matching module is proposed to detect intrusions based on both intra-pattern and inter-pattern anomalies. We test this idea using the data sets obtained from the University of New Mexico. The experimental results indicate that our scheme detects significantly more anomalies than the scheme presented in (Wespi et al., 2000) while maintaining a very low false alarm rate.