Isolating failing test cases: A comparative experimental study of clustering techniques

Researchers have applied cluster analysis onto execution profiles induced by test cases in order to solve problems in the area of software testing and analysis. The employed clustering techniques varied, and no study was conducted to rate these techniques in terms of their effectiveness in this specific domain. This work aims at doing so experimentally. Specifically, given test sets comprising passing and failing test cases, we measure the performance of each technique at isolating the failing test cases from the passing cases. The study included one technique from each of the six main families of clustering algorithms. Our results suggested the following ranking of the evaluated techniques from best to worst: DBSCAN, K-Means, Agglomerative-AGNES and WaveCluster, Fuzzy-FCM, and K-Subspace.