High performance networking for supercomputers

This paper provides a case study on how a high performance network was designed and implemented for the world´s fastest supercomputer, i.e. the ASCI Red at Sandia National Laboratories. It presents the methodology that increased the throughput of ASCI Red from less than 1 Megabyte/sec. to 60 Megabytes/sec. This methodology is based upon viewing the network as a whole and not merely disjoint components that are connected together. The case study analyzes the many variables which affect the throughput, including network technology, transport media, distance between nodes, memory buffer size, maximum transfer unit size, protocol, achieving parallelism, and IP stack tuning. This paper begins by describing some of the special purpose machines which comprise the Sandia National Laboratories High Performance Computing Environment. It then discusses how these machines and the above variables were optimized in an integrated network.