A Distributed Dataflow Model for Task-Uncoordinated Parallel Program Execution

High Performance Computing (HPC) systems now consist of many thousands of individual servers. While relatively scalable and cost effective, these systems suffer from a complexity of scale that will not improve with increasing machine size. It will become increasingly difficult, if not impossible, for HPC systems to maintain node availability long enough for any type of worthwhile scientific calculations to be performed. Existing execution and programming models, which are dependent on guaranteed hardware reliability, are not well suited to future distributed memory parallel systems where hardware reliability cannot be guaranteed. We propose a distributed dataflow execution model which utilizes a distributed dictionary for data memoization, allowing each parallel task to schedule instructions without direct inter-task coordination. We provide a description of the proposed execution model, including program formulation and autonomous dataflow task selection. Experiments performed demonstrate the proposed model´s ability to automatically distribute work across tasks, as well as the proposed model´s ability to scale in both shared memory and distributed memory.