Combining static analysis and dynamic learning to build accurate intrusion detection models

Anomaly detection based on monitoring of sequences of system calls has been shown to be an effective method for detection of previously unseen, potentially damaging attacks on hosts. This paper presents a new model for profiling normal program behavior for use in detection of intrusions that change application execution flow. This model is compact and efficient to operate and can be acquired using a combination of static analysis and dynamic learning. Our model (hybrid push down automata, HPDA) incorporates call stack information in the automata model and effectively captures the control flow of a program. Several important properties of the model are based on a unique correspondence relation between addresses and instructions within the model. These properties allow the HPDA to be acquired by dynamic analysis of an audit of the call stack log. Our strategy is to use static analysis to acquire a base model and then to use dynamic learning as a supplement to capture those aspects of behavior that are difficult to capture with static analysis due to techniques commonly used in modern programming environments. The model created by this combination method is shown to have a higher detection capability than models acquired by static analysis alone and a lower false positive rate than models acquired by dynamic learning alone.