Data mining for security applications: Mining concept-drifting data streams to detect peer to peer botnet traffic

The presentation first provide an overview for data mining for security applications and then discuss our research to the botnet problem which follows from an important observation that network traffic (as well as botnet traffic) is a continuous flow of data stream. Conventional data mining techniques are not directly applicable to stream data because of two vital problems associated with them: potentially infinite in length, and concept drift. We propose a technique that can efficiently handle both problems. Our main focus is to adapt three major data mining techniques: classification, clustering, and outlier detection to handle stream data. Our preliminary study on the development of new stream classification techniques for P2P bothnet detection has generated encouraging results. In addition to botnet detection, we also discuss our research on data mining for malicious code detection and intrusion detection.