Detecting attacks that exploit application-logic errors through application-level auditing

Host security is achieved by securing both the operating system kernel and the privileged applications that run on top of it. Application-level bugs are more frequent than kernel-level bugs, and, therefore, applications are often the means to compromise the security of a system. Detecting these attacks can be difficult, especially in the case of attacks that exploit application-logic errors. These attacks seldom exhibit characterizing patterns as in the case of buffer overflows and format string attacks. In addition, the data used by intrusion detection systems is either too low-level, as in the case of system calls, or incomplete, as in the case of syslog entries. This paper presents a technique to enforce nonbypassable, application-level auditing that does not require the recompilation of legacy systems. The technique is implemented as a kernel-level component, a privileged daemon, and an offline language tool. The technique uses binary rewriting to instrument applications so that meaningful and complete audit information can be extracted. This information is then matched against application-specific signatures to detect attacks that exploit application-logic errors. The technique has been successfully applied to detect attacks against widely-deployed applications, including the Apache Web server and the OpenSSH server.