A Programming Framework for Incremental Data Distribution in Iterative Applications

Author

Chan, Philip ; Abramson, David

Author_Institution

Caulfield Sch. of Inf. Technol., Monash Univ., Caulfield East, VIC, Australia

fYear

2008

fDate

10-12 Dec. 2008

Firstpage

244

Lastpage

251

Abstract

Successful HPC over desktop grids and non-dedicated NOWs is challenging, since good performance is difficult to achieve due to dynamic workloads. On iterative data-parallel applications, this is addressed by dynamic data distribution. However, current approaches migrate an application from one distribution to another in one single phase, which can impact performance. In this paper, we present D3-ARC, a programming framework to support adaptive and incremental data distribution, so that data migration takes place over several successive iterations. D3-ARC consists of a runtime system and an API for specifying the distribution of arrays as well as how data redistribution takes place. We demonstrate how D3-ARC can be used to develop an incremental strategy for data distribution in a Poisson solver, utilising a runtime feedback mechanism to determine how much data to migrate during each iteration.

Keywords

application program interfaces; data handling; API; D3-ARC; Poisson solver; incremental data distribution; programming framework; runtime feedback mechanism; Australia; Distributed computing; Distributed processing; Dynamic programming; Feedback; Information technology; Laboratories; Parallel programming; Peer to peer computing; Runtime; Dynamic Data Redistribution; Incremental Data Distribution; Iterative Applications; Message-Passing Interface; Parallel Programming Frameworks;

fLanguage

English

Publisher

ieee

Conference_Titel

Parallel and Distributed Processing with Applications, 2008. ISPA '08. International Symposium on

Conference_Location

Sydney, NSW

Print_ISBN

978-0-7695-3471-8

Type

conf

DOI

10.1109/ISPA.2008.105

Filename

4725155