Title :
Optimized Routing for Large-Scale InfiniBand Networks
Author :
Hoefler, Torsten ; Schneider, Timo ; Lumsdaine, Andrew
Author_Institution :
Open Syst. Lab., Indiana Univ., Bloomington, IN, USA
Abstract :
Point-to-point metrics, such as latency and bandwidth, are often used to characterize network performance with the consequent assumption that optimizing for these metrics is sufficient to improve parallel application performance. However, these metrics can only provide limited insight into application behavior because they do not fully account for effects, such as network congestion, that significantly influence overall network performance. Because many high-performance networks use deterministic oblivious routing, one such effect is the choice of routing algorithm. In this paper, we analyze and compare practical and theoretical aspects of different routing algorithms that are used in today\´s large-scale networks. We show that widely-used theoretical metrics, such as edge-forwarding index or bisection bandwidth, are not accurate predictors for average network bandwidth. Instead, we introduce an intuitive metric, which we call "effective bisection bandwidth" to characterize quality of different routing algorithms. We present a simple algorithm that globally balances routes and therefore improves the effective bandwidth of the network. Compared to the best algorithm in use today, our new algorithm shows an improvement in effective bisection bandwidth of 40% on a 724-endpoint InfiniBand cluster.
Keywords :
radio links; telecommunication network routing; InfiniBand networks routing; bisection bandwidth; edge-forwarding index; large-scale networks; network congestion; point-to-point metrics; routing algorithm; Adaptive algorithm; Bandwidth; Clustering algorithms; Delay; Intelligent networks; Large-scale systems; Network topology; Open systems; Routing; Telecommunication traffic; InfiniBand; clusters; high performance computing; message passing; networks; routing;
Conference_Titel :
High Performance Interconnects, 2009. HOTI 2009. 17th IEEE Symposium on
Conference_Location :
New York, NY
Print_ISBN :
978-0-7695-3847-1
Electronic_ISBN :
1550-4794
DOI :
10.1109/HOTI.2009.9