Combining Grid and Cloud Resources by Use of Middleware for SPMD Applications

Distributed computing environments have evolved from in-house clusters to Grids and now Cloud platforms. We, as others, provide HPC benchmarks results over Amazon EC2 that show a lower performance of Cloud resources compared to private resources., So, it is not yet clear how much of impact Clouds will have in high performance computing (HPC). But hybrid Grid/Cloud computing may offer opportunities to increase overall applications performance, while benefiting from in-house computational resources extending them by Cloud ones only whenever needed. In this paper, we advocate the usage of Proactive, a well established middleware in the grid community, for mixed Grid/Cloud computing, extended with features to address Grid/Cloud issues with little or, no effort for application developers. We also introduce a framework, developed in the context of the Disco Grid project, based upon the Proactive middleware to couple HPC domain-decomposition SPMD applications in heterogeneous multi-domain environments. Performance results, coupling Grid and Cloud resources for the execution of such, kind of highly communicating and processing intensive applications, have shown an overhead of about 15%, which is a non-negligible value, but lower enough to consider using such environments to achieve a better cost-performance trade-off than using exclusively Cloud resources.