Metamorphic malware detection using Linear Discriminant Analysis and Graph Similarity

The most common malware detection approaches which are based on signature matching and are not sufficient for metamorphic malware detection, since virus kits and metamorphic engines can produce variants with no resemblance to one another. Metamorphism provides an efficient way for eluding malware detection software kits. Code obfuscation methods like dead-code insertion are also widely used in metamorphic malware. In order to address the problem of detecting mutated generations, we propose a method based on Opcode Graph Similarity (OGS). OGS tries to detect metamorphic malware using the similarity of opcode graphs. In this method, all nodes and edges have a respective effect on classification, but in the proposed method, edges of graphs are pruned using Linear Discriminant Analysis (LDA). LDA is based on the concept of searching for a linear combination of predictors that best separates two or more classes. Most distinctive edges are identified with LDA and the rest of edges are removed. The metamorphic malware families considered here are NGVCK and metamorphic worms that we denote these worms as MWOR. The results show that our approach is capable of classifying metamorphosed instances with no or minimum false alarms. Also, our proposed method can detect NGVCK and MWOR with high accuracy rate.