DocumentCode :
560145
Title :
Simplified parallel domain traversal
Author :
Kendall, Wesley ; Wang, Jingyuan ; Allen, Melissa ; Peterka, Tom ; Huang, Jian ; Erickson, David
Author_Institution :
Dept. of Electr. Eng. & Comput. Sci., Univ. of Tennessee, Knoxville, TN, USA
fYear :
2011
fDate :
12-18 Nov. 2011
Firstpage :
1
Lastpage :
11
Abstract :
Many data-intensive scientific analysis techniques require global domain traversal, which over the years has been a bottleneck for efficient parallelization across distributed- memory architectures. Inspired by MapReduce and other simplified parallel programming approaches, we have designed DStep, a flexible system that greatly simplifies efficient parallelization of domain traversal techniques at scale. In order to deliver both simplicity to users as well as scalability on HPC platforms, we introduce a novel two-tiered communication architecture for managing and exploiting asynchronous communication loads. We also integrate our design with advanced parallel I/O techniques that operate directly on native simulation output. We demonstrate DStep by performing teleconnection analysis across ensemble runs of terascale atmospheric CO2 and climate data, and we show scalability results on up to 65,536 IBM BlueGene/P cores.
Keywords :
data analysis; distributed memory systems; parallel processing; parallel programming; DStep; HPC platforms; MapReduce; asynchronous communication loads; climate data; data-intensive scientific analysis techniques; distributed-memory architectures; parallel I-O techniques; parallel domain traversal; parallel programming approaches; teleconnection analysis; terascale atmospheric; two-tiered communication architecture; Analytical models; Atmospheric modeling; Computer architecture; Data visualization; Message systems; Protocols; Scalability; Atmospheric Ensemble Analysis; Data-Intensive Analysis; Parallel Particle Tracing; Parallel Processing;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
High Performance Computing, Networking, Storage and Analysis (SC), 2011 International Conference for
Conference_Location :
Seatle, WA
Electronic_ISBN :
978-1-4503-0771-0
Type :
conf
Filename :
6114407
Link To Document :
بازگشت