DocumentCode :
3117258
Title :
Nimrod/K: Towards massively parallel dynamic Grid workflows
Author :
Abramson, David ; Enticott, Colin ; Altinas, Ilkay
Author_Institution :
Fac. of Inf. Technol., Monash Univ., Clayton, VIC, Australia
fYear :
2008
fDate :
15-21 Nov. 2008
Firstpage :
1
Lastpage :
11
Abstract :
A challenge for Grid computing is the difficulty in developing software that is parallel, distributed and highly dynamic. Whilst there have been many general purpose mechanisms developed over the years, Grid programming still remains a low level, error prone task. Scientific workflow engines can double as programming environments, and allow a user to compose dasiavirtualpsila Grid applications from pre-existing components. Whilst existing workflow engines can specify arbitrary parallel programs, (where components use message passing) they are typically not effective with large and variable parallelism. Here we discuss dynamic dataflow, originally developed for parallel tagged dataflow architectures (TDAs), and show that these can be used for implementing Grid workflows. TDAs spawn parallel threads dynamically without additional programming. We have added TDAs to Kepler, and show that the system can orchestrate workflows that have large amounts of variable parallelism. We demonstrate the system using case studies in chemistry and in cardiac modelling.
Keywords :
grid computing; message passing; multi-threading; natural sciences computing; TDA; distributed software; message passing; parallel dynamic grid computing; parallel program; parallel software; parallel thread; scientific workflow engine; tagged dataflow architecture; Application software; Chemistry; Dynamic programming; Engines; Grid computing; Message passing; Parallel processing; Parallel programming; Programming environments; Yarn;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
High Performance Computing, Networking, Storage and Analysis, 2008. SC 2008. International Conference for
Conference_Location :
Austin, TX
Print_ISBN :
978-1-4244-2834-2
Electronic_ISBN :
978-1-4244-2835-9
Type :
conf
DOI :
10.1109/SC.2008.5215726
Filename :
5215726
Link To Document :
بازگشت