Data mining on PC cluster connected with storage area network: its preliminary experimental results

Personal computer/workstation (PC/WS) clusters have become a hot research topic in the field of parallel and distributed computing. They are considered to play an important role as a large scale computer system, such as large server sites and/or high performance parallel computers, because of their good scalability and cost performance ratio. In the viewpoint of applications, data intensive applications such as data mining and ad-hoc query processing in databases are considered very important for massively parallel processors, in addition to the conventional scientific calculation. Thus, investigating the feasibility of such applications on a PC cluster is meaningful. A PC cluster connected with a storage area network (SAN) is built and evaluated. For disk-to-disk copy operation, SAN clusters are much better than LAN clusters. A data mining application is implemented on the cluster. This application requires iterative scans of shared disks, which degrade the execution performance due to I/O-bottleneck. In order to resolve the problem, a dynamic data copy method is proposed and evaluated. This method prevents the performance degradation caused by shared disk bottleneck in SAN clusters