Detecting Compromised VM via Application-Aware Anomaly Detection

Nowadays, the Infrastructure as a Service (IaaS) cloud has become the new target of attackers and the security of virtual machine (VM) in cloud is attracting more and more attention. In this paper, we propose to use virtualization to combine information of system level with that of network level and monitor the behavior of VM at the granularity of process. Based on Xen hypervisor, we implement this approach and develop APPLICATION-AWARE ANOMALY DETECTION SYSTEM (AADS) to detect anomalous behavior in VM. Experimental results show that our correlated approach performs better than that use features from only system level or network level.