DocumentCode :
1926880
Title :
Design alternatives for implementing fence synchronization in MPI-2 one-sided communication for InfiniBand clusters
Author :
Santhanaraman, G. ; Gangadharappa, T. ; Narravula, S. ; Mamidala, A. ; Panda, D.K.
Author_Institution :
Dept. of Comput. Sci. & Eng., Ohio State Univ., Columbus, OH, USA
fYear :
2009
fDate :
Aug. 31 2009-Sept. 4 2009
Firstpage :
1
Lastpage :
9
Abstract :
Scientific computing has seen an immense growth in recent years. The Message Passing Interface (MPI) has become the de-facto standard for parallel programming model for distributed memory systems. As the system scale increases, application writers often try to increase the overlap of computation and communication. The MPI-2 standard expanded MPI to include one-sided communication semantics that has the potential for overlapping computation with communication. In this model synchronization between processes needs to be done explicitly to ensure completion before using the data. Fence is one of the mechanisms of providing such synchronization in the one-sided model. In this paper, we study a set of different alternatives for designing the fence synchronization mechanisms. We analyze the various trade-offs of these designs on networks like InfiniBand that provide Remote Direct Memory Access (RDMA) capabilities. We propose a novel design for implementing fence synchronization that uses RDMA write with Immediate mechanism (Fence-Imm-RI) provided by InfiniBand networks. We then characterize the performance of different designs with various one-sided communication pattern microbenchmarks for both latency as well as overlap capability. The new Fence-Imm-RI scheme performs the best in scenarios that require low synchronization overhead as well as good overlap capability (close to 90% overlap for large messages) as opposed to the other designs that can provide either low synchronization overhead or good overlap capability.
Keywords :
distributed memory systems; file organisation; message passing; parallel programming; synchronisation; Fence-Imm-RI scheme; InfiniBand clusters; RDMA write with Immediate mechanism; distributed memory systems; fence synchronization; message passing interface; one-sided communication semantics; parallel programming model; remote direct memory access; Application software; Communication standards; Computer science; Delay; Design engineering; Message passing; Middleware; Parallel programming; Read-write memory; Scientific computing;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Cluster Computing and Workshops, 2009. CLUSTER '09. IEEE International Conference on
Conference_Location :
New Orleans, LA
ISSN :
1552-5244
Print_ISBN :
978-1-4244-5011-4
Electronic_ISBN :
1552-5244
Type :
conf
DOI :
10.1109/CLUSTR.2009.5289200
Filename :
5289200
Link To Document :
بازگشت