Runtime Dynamic Structural Changes of Scientific Workflows in Clouds

Existing Scientific Workflow Management Systems (i.e. SWfMS) effectively support workflows that do not need dynamic changes at runtime. SWfMS execute workflows by providing process management, provenance data and distributed execution on clusters and clouds. However, the support for dynamic changes in workflows is still an open, yet important, problem. For example, when the program associated to an activity of the workflow is taking more time than expected to produce results or if the results do not comply with some quality criteria, the scientist may want to try an alternative algorithm implementation. However, scientists may not want to re-execute the entire workflow for each change they make in the workflow structure. Alternatively, changing the structure of the workflow dynamically (i.e. change the programs associated with workflow activities) can improve the workflow execution without restarting the execution. In this paper, we propose DynAdapt, a dynamic approach that allows scientist to perform structural changes of the workflow specification at runtime. We evaluated DynAdapt using SciPhy, a large-scale bioinformatics workflow, and results show up to 40% of execution improvement.