Mining Frequent Closed Itemsets in Large Databases by Hierarchical Partitioning

The mining of frequent itemsets has been extensively studied in data mining, and many methods have been proposed for this problem. However, mining all the frequent itemsets will lead to a huge number of itemsets and numerous redundant association rules. Fortunately, this problem can be cured by mining only frequent closed itemsets (FCIs), which results in a much smaller number of itemsets. Nevertheless, it is still difficult to find FCIs when the database becomes too large to allow a memory-resident representation. In this paper, a methodology called hierarchical partitioning is proposed for dividing the database into a set of multi-leveled sub-databases of manageable sizes to fit into memory. The advantage of hierarchical partitioning is that the FCIs can be found directly from sub-databases without rescanning the original database for support and subset checking.