Polymorphic malicious executable scanner by API sequence analysis

The proliferation of malware (viruses, Trojans, and other malicious code) in recent years has presented a serious threat to enterprises, organizations, and individuals. Polymorphic (or variant versions of) computer viruses are more complex and difficult than their original versions to detect, often requiring antivirus companies to spend much time to create the routines needed to catch them. In this paper, we propose a new approach for detecting polymorphic malware in the Windows platform. Our approach rests on an analysis based on the Windows API calling sequence that reflects the behavior of a piece of particular code. The analysis is carried out directly on the PE (portable executable) code. It is achieved in two major steps: construct the API calling sequences for both the known virus and the suspicious code, and perform a similarity measurement between the two sequences after a sequence realignment operation is done. Favorable experimental results are obtained and presented.