e-NeXSh: achieving an effectively non-executable stack and heap via system-call policing

We present e-NeXSh, a novel security approach that utilises kernel and LIBC support for efficiently defending systems against process-subversion attacks. Such attacks exploit vulnerabilities in software to override its program control-flow and consequently invoke system calls, causing out-of-process damage. Our technique defeats such attacks by monitoring all LIBC function and system-call invocations, and validating them against process-specific information that strictly prescribes the permissible behaviour for the program (unlike general sandboxing techniques that require manually maintained, explicit policies, we use the program code itself as a guideline for an implicit policy). Any deviation from this behaviour is considered malicious, and we halt the attack, limiting its damage to within the subverted process. We implemented e-NeXSh as a set of modifications to the Linux-2.4.18-3 kernel and a new user-space shared library (e-NeXSh.so). The technique is transparent, requiring no modifications to existing libraries or applications. e-NeXSh was able to successfully defeat both code-injection and LIBC-based attacks in our effectiveness tests. The technique is simple and lightweight, demonstrating no measurable overhead for select UNIX utilities, and a negligible 1.55% performance impact on the Apache Web server