Rule generation with the pattern repository

Efficient algorithms for mining frequent patterns are crucial to many tasks in data mining. Since the Apriori algorithm was proposed in 1994, there have been several methods developed to improve its performance. However, most still adopt its candidate set generation-and-test approach. In addition, many methods do not generate all frequent patterns, making them inadequate to derive all association rules. The calculation of association rules from raw itemsets using Apriori is an intractable problem. By using a new structure called a pattern repository, the same rules can be derived in linear-time proportional to the number of unique items found. If the selected rules are all we need, the calculation can give results in real-time. In addition, the calculation can easily be divided into subsets for distributed processing and large datasets can be stored on disk that adds to the I/O overhead, but still offers a linear time calculation of rules.