DocumentCode :
1924837
Title :
The Power and Challenges of Transformative I/O
Author :
Manzanares, Adam ; Bent, John ; Wingate, Meghan ; Gibson, Garth
Author_Institution :
Los Alamos Nat. Lab., Los Alamos, NM, USA
fYear :
2012
fDate :
24-28 Sept. 2012
Firstpage :
144
Lastpage :
154
Abstract :
Extracting high data bandwidth and metadata rates from parallel file systems is notoriously difficult. User workloads almost never achieve the performance of synthetic benchmarks. The reason for this is that real-world applications are not as well-aligned, well-tuned, or consistent as are synthetic benchmarks. There are at least three possible ways to address this challenge: modification of the real-world workloads, modification of the underlying parallel file systems, or reorganization of the real-world workloads using Tran formative middleware. In this paper, we demonstrate that Tran formative middleware is applicable across a large set of high performance computing workloads and is portable across the three major parallel file systems in use today. We also demonstrate that our transformative middleware layer is capable of improving the write, read, and metadata performance of I/O workloads by up to 150x, 10x, and 17x respectively, on workloads with processor counts of up to 65,536.
Keywords :
file organisation; meta data; middleware; parallel processing; high performance computing workload; metadata rate; parallel file system; synthetic benchmark; tranformative middleware; transformative I/O; user workload; Aggregates; Bandwidth; Concurrent computing; Containers; Indexes; Libraries; Middleware; Checkpointing; High Performance Computing; Parallel Computing; Parallel File Systems & I/O;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Cluster Computing (CLUSTER), 2012 IEEE International Conference on
Conference_Location :
Beijing
Print_ISBN :
978-1-4673-2422-9
Type :
conf
DOI :
10.1109/CLUSTER.2012.86
Filename :
6337775
Link To Document :
بازگشت