An assessment of Gigabit Ethernet as cluster interconnect

With the advent of Gigabit Ethernet network technology, a new contender for the next-generation cluster interconnect is on the horizon. We benchmark Gigabit Ethernet and compare it to other cluster interconnects, namely Fast Ethernet, Myrinet and Scalable Coherent Interface. For a meaningful comparison, benchmark experiments are carried out at two levels: TCP/IP networking and MPI parallel programming. Using high-end PCs (Pentium III 450 MHz) and standard system software (Linux and MPICH), our results show that with Gigabit Ethernet, end-to-end throughputs of up to 44 MBytes per second can be achieved. A change from Fast Ethernet to Gigabit Ethernet resulted in performance gains of up to a factor of 5.05 for ftp, up to a factor of 3.10 for MPI point-to-point communication and up to a factor of 2.58 for some NAS parallel benchmarks, depending on communication granularity and message size. Further performance gains seem to require improved protocol processing and node hardware (memory bandwidth, file system, PCI bus).