Facilitating Scientific Workflow Configuration with Parameterized Workflow Skeletons

This paper describes an operator for configuring scientific workflows that facilitates the process of assigning workflow activities to cloud resources. In general, modeling and configuring scientific workflows is complex and error-prone, because workflows are built of highly parallel patterns comprising huge numbers of tasks. Reusing tested patterns as building blocks avoids repeating errors. Workflow skeletons are parametrizable building blocks describing such patterns. Hence, scientists have a means to reuse validated parallel constructs for rapidly defining their in-silico experiments. Often, configurations of data parallel patterns are generated automatically. However, for many task parallel patterns each task needs to be configured manually. In frameworks like MapReduce, scientists have no control of how tasks are assigned to cloud resources. What is the strength of such patterns, may lead to unnecessary data transfers in other patterns. Workflow Skeletons facilitate the configuration by providing an operator that accepts parameters, this allows for scalable configurations saving time and cost by allocating cloud resources just in time. In addition, this configuration operator helps to define configurations that avoid unnecessary data transfers.