Towards Terabit/s Systems: Performance Evaluation of Multi-Rail Systems

We present a novel multi-rail approach that is necessary in order for future E-Science applications to effectively exploit Terabit/s networks. The multi-rail approach consists of creating parallel "rails" through every aspect of an end-system: from processing on the multiple cores, generation of multiple application data flows, and streaming over multiple-lanes, multi-wavelength NICs connected via a parallel interconnect. In this paper, we evaluate the end-systems parameters that impact the efficiency of multi-rail systems such as interrupt, memory, thread, and core affinities. These evaluations are tested on the ability of single cluster nodes to achieve TCP and UDP throughput at 10 Gbps and 20 Gbps rates. We analyze the additive effects of the parameters -a key property for achieving scalable performance towards Terabits/s. Thread and interrupt affinity together were found to have an additive effect and play a critical role in achieving a throughput of 20 Gbps.