Mining Non-derivable Hypercliques

Hyper cliques have been successfully applied in a number of applications, e.g. clustering and noise removal. A hyper clique is an item set containing items that are strongly correlated with each other. Even though hyper cliques have been shown to handle datasets with skewed support distribution and low support threshold well, they might still face problems for dense datasets and lower h-confidence threshold. In this paper, we propose a new pruning method based on combining hyper cliques and Non-Derivable Item sets (NDIs) in order to substantially reduce the amount of generated hyper clique sets. Specifically, we propose a new collection of hyper cliques, called Non-Derivable Hyper cliques (NDHC), and present an efficient algorithm to mine these sets, called NDHC Miner. The proposed NDHC collection is a loss less representation of hyper cliques, i.e., given the item sets in NDHC, we can generate the complete collection of hyper cliques and their support, without additional scanning of the dataset. We experimentally compare NDHC with Hyper cliques (HC), as well as another condensed representation of hyper cliques, maximal hyper cliques (MHP). Our experiments show that the NDHC collection offers substantial advantages over HC, and even MHP, especially for dense datasets and lower h-confidence threshold values.