Evolving multi-objective strategies for task allocation of scientific workflows on public clouds

With the increase in deployment of scientific application on public and private clouds, the allocation of workflow tasks to specific cloud instances to reduce runtime and cost has emerged as an important challenge. The allocation of scientific workflows on public clouds can be described through a variety of perspectives and parameters and has been proved to be NP-complete. This paper presents an optimization framework for task allocation on public clouds. We present a solution that considers important parameters such as workflow runtime, communication overhead, and overall execution cost. Our multi-objective optimization framework builds on a simple and extensible cost model and uses a heuristic to determine the optimal number of cloud instances to be used. Using the Amazon Elastic Compute Cloud (EC2) and Amazon Simple Storage Service (S3) as an example, we show how our optimization heuristics lead to significantly better strategies than other state-of-the-art approaches. Specifically, our single-objective optimization is slightly better than a simple heuristic and a particle swarm optimization approach for small workflows, and achieves significant improvements for larger workflows. In a similar manner, our multi-objective optimization obtains similar results to our single-objective optimization for small-size workflows, and achieves up to 80% improvement for large-size workflows.