Transparent Optimization of Parallel File System I/O via Standard System Tool Enhancement

Author

Kolano, Paul Z.

Author_Institution

NASA Adv. Supercomput. Div., NASA Ames Res. Center, Moffett Field, CA, USA

fYear

2013

fDate

20-24 May 2013

Firstpage

1963

Lastpage

1970

Abstract

Standard system tools employed by users on a daily basis do not take full advantage of parallel file system I/O bandwidth and do not understand associated idiosyncrasies such as Lustre striping. This can lead ton on-optimal utilization of both the user´s time and system resources. This paper describes a set of modifications made to existing tools that increase parallelism and automatically handle striping. These modifications result insignificant performance gains in a transparent manner with maximum speedups of 27×, 15×, and 31× for parallelized cp, tar creation, and tar extraction, respectively.

Keywords

input-output programs; optimisation; parallel processing; Lustre striping; nonoptimal utilization; parallel file system I/O bandwidth; parallelized cp; standard system tool enhancement; tar creation; tar extraction; transparent optimization; Bandwidth; Instruction sets; Message systems; Optimization; Performance gain; Standards; high performance system tools; lustre; parallel file systems;

fLanguage

English

Publisher

ieee

Conference_Titel

Parallel and Distributed Processing Symposium Workshops & PhD Forum (IPDPSW), 2013 IEEE 27th International

Conference_Location

Cambridge, MA

Print_ISBN

978-0-7695-4979-8

Type

conf

DOI

10.1109/IPDPSW.2013.192

Filename

6651100