Proactive malware collection and classification system: How to collect and classify useful malware samples?

To understand malware behaviors, collecting and classifying malware samples is a critical issue for system security researchers. This paper aims to develop Proactive Malware Collection and Classification System (PMCCS), which consists of Proactive Malware Collection Unit (PMCU) and Automatic Malware Classification Unit (AMCU). To collect useful samples, PMCU uses P2P software actively search suspicious samples, such as software crack tools. During a 3-year period, PMCU has collected 42300 samples. To automatically classify useful samples, AMCU uploads suspicious samples to VirusTotal, a free online virus scanner. Based on VirusTotal scanning results, 11600 suspicious samples have been alerted at least once by AntiVirusWare (AVW) and 70% of these samples are Trojans and Virus tools, which are usually threatening malwares. Moreover, these suspicious 11600 samples are classified into: Blacklist with high suspiciousness; Ambitious list with moderate suspiciousness; Whitelist with low suspiciousness. Blacklist can be used to evaluate the performance of AVW based on False Negative (FN). On the other hand, Whitelist can be used to evaluate the performance of AVW based on False Positive (FP). From Blacklist and Whitelist, AMCU selects useful malwares, which triggering high counts of FN and FP against AVW.