Dynamic malware detection using registers values set analysis

The number of Malicious files increase every day because of existing open source malware and obfuscation techniques. It means that traditional signature-based techniques are not adequate for detecting new variant of malware. Researchers and anti malware companies recently focus on more advanced protection which needs influential pattern extraction techniques. In this paper, a novel method is proposed based on similarities of binaries behaviors. At first, Run-time behavior of the binary files are found and logged in a controlled environment tool which is developed in-house. The approach assumes that behavior of each binary can be represented by the values of memory contents in its run-time. That is, values stored in different registers while the malware is running in the controlled environment can be a distinguishing factor to set it apart from those of benign programs. Then, the register values for each Application Programming Interface (API) call are extracted before and after API is invoked. After that, we traced the distribution and changes of registers values throughout the executable file and created a vector for each of the values of EAX, EBX, EDX, EDI, ESI and EBP registers. With comparing the similarity measures between old and unseen malware vectors, we detected 98% of unseen samples and with 2.9% false positive.