Host-based intrusion detection by monitoring Windows registry accesses

We propose a host-based intrusion detection system for Microsoft Windows. The proposed system detects attacks on a host machine by monitoring anomalous accesses to the Windows registry. First, a model of normal registry behavior is trained for a host and then this model is used to detect abnormal registry accesses. The system trains a normal model using data that contains no attacks and then checks each access to the registry to determine whether or not the behavior is abnormal and corresponds to an attack. A new approach to register anomaly detection (RAD) is proposed in the meaning of model generator and anomaly detector. A self organizing map (SOM), a type of artificial neural network model, is used as an anomaly detection algorithm. The system is trained on a set of normal registry accesses using SOM algorithm and then it is used to detect the behavior of malicious software. The results of this study show that the proposed system is effective in detecting the behavior of malicious software and has a low rate of false alarms compared to other host-based intrusion detection systems.