Title :
Investigation of distributive data mining calculating architecture
Author :
Fang, Ying-Wu ; Yu-Mei Huang ; Zhang, Guang-Peng ; Wang, Yi ; Wang, Shao-Long
Author_Institution :
Sch. of Mech. & Precision Instrum. Eng., Xi´´an Univ. of Technol., China
Abstract :
A distributive calculating architecture is presented to realize data mining efficiently. The architecture is a hierarchical computational method from the conception, which stores the information in every sub-node with the ideas of database partition, a united center distribute unit can be responsible for the collecting and maintenance of the information in every sub-node, by the scan of database, information is distributed to different nodes, this architecture can maintain a global set enumerate tree, the local large item-sets can be constructed by using any effective algorithm, it mainly solves the problem of highly effective data distribution and data skew, the detailed explaining and theoretical proving of the calculating architecture is given, and how to solve the data skew problem highly and effectively is also discussed in this paper. The partial implementation of this algorithm shows the correctness and feasibility, the calculating architecture can be used for distribute database and most applicable for distribute calculation, which can be used in highly and effectively data mining in distributive and parallel environment.
Keywords :
data mining; data warehouses; distributed databases; information storage; problem solving; data distribution; data skew problem solving; distributive calculating architecture; distributive data mining; distributive environment; hierarchical computational method; information collection; information maintenance; information storage; parallel environment; Association rules; Computer architecture; Data engineering; Data mining; Distributed computing; Distributed databases; Electronic mail; Information analysis; Instruments; Partitioning algorithms;
Conference_Titel :
Machine Learning and Cybernetics, 2004. Proceedings of 2004 International Conference on
Print_ISBN :
0-7803-8403-2
DOI :
10.1109/ICMLC.2004.1382041