Title :
Multithreaded Two-Phase I/O: Improving Collective MPI-IO Performance on a Lustre File System
Author :
Tsujita, Y. ; Yoshinaga, Koji ; Hori, A. ; Sato, Mitsuhisa ; Namiki, Mitaro ; Ishikawa, Yozo
Author_Institution :
RIKEN AICS, Wako, Japan
Abstract :
ROMIO, a representative MPI-IO implementation, has been widely used in recent large-scale parallel computations. The two-phase I/O optimization scheme of ROMIO improves I/O performance for non-contiguous access patterns, however, this scheme still has room to improve performance to make it suitable for recent data-intensive computing. We propose overlapping data exchange operations with file I/O operations by using a multithreaded scheme to achieve further I/O throughput improvement. We show up to 60% improvement by the multithreaded two-phase I/O relative to the original two-phase I/O in performance evaluation of collective write operations on a Lustre file system of a Linux PC cluster.
Keywords :
Linux; application program interfaces; message passing; multi-threading; optimisation; Linux PC cluster; Lustre file system; ROMIO; collective MPI-IO performance; data-intensive computing; multithreaded two-phase I/O optimization scheme; overlapping data exchange operations; performance evaluation; representative MPI-IO implementation; Benchmark testing; Educational institutions; Electronic mail; Instruction sets; Libraries; Performance evaluation; Throughput; MPI-IO; ROMIO; collective I/O; multithreading; two-phase I/O;
Conference_Titel :
Parallel, Distributed and Network-Based Processing (PDP), 2014 22nd Euromicro International Conference on
Conference_Location :
Torino
DOI :
10.1109/PDP.2014.46
