DocumentCode :
3248256
Title :
Towards an understanding of the performance of MPI-IO in Lustre file systems
Author :
Logan, Jeremy ; Dickens, Phillip
Author_Institution :
Dept. of Comput. Sci., Univ. of Maine, Orono, ME
fYear :
2008
fDate :
Sept. 29 2008-Oct. 1 2008
Firstpage :
330
Lastpage :
335
Abstract :
Lustre is becoming an increasingly important file system for large-scale computing clusters. The problem, however, is that many data-intensive applications use MPI-IO for their I/O requirements, and MPI-IO performs poorly in a Lustre file system environment. While this poor performance has been well documented, the reasons for such performance are currently not well understood. Our research suggests that the primary performance issues have to do with the assumptions underpinning most of the parallel I/O optimizations implemented in MPI-IO, which do not appear to hold in a Lustre environment. Perhaps the most important assumption is that optimal performance is obtained by performing large, contiguous I/O operations. However, the research results presented in this poster show that this is often the worst approach to take in a Lustre file system. In fact, we found that the best performance is often achieved when each process performs a series of smaller, non-contiguous I/O requests. In this poster, we provide experimental results supporting these non-intuitive ideas, and provide alternative approaches that significantly enhance the performance of MPI-IO in a Lustre file system.
Keywords :
application program interfaces; file organisation; message passing; Lustre file system; MPI-IO; large-scale computing cluster; Aggregates; Analytical models; Application software; Computational modeling; Computer science; File systems; Image analysis; Large-scale systems; Power system management; Scalability;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Cluster Computing, 2008 IEEE International Conference on
Conference_Location :
Tsukuba
ISSN :
1552-5244
Print_ISBN :
978-1-4244-2639-3
Electronic_ISBN :
1552-5244
Type :
conf
DOI :
10.1109/CLUSTR.2008.4663791
Filename :
4663791
Link To Document :
بازگشت