Extending Scientific Workflow Systems to Support MapReduce Based Applications in the Cloud

Author

Gugnani, Shashank ; Kiss, Tamas

Author_Institution

Center for Parallel Comput., Univ. of Westminster, London, UK

fYear

2015

Firstpage

16

Lastpage

21

Abstract

Cloud Computing has gained a lot of popularity in recent years because of the flexibility that it offers. In addition, there seems to be a rising interest in combining Parallel Computing, Cloud Computing and Big Data to create large scale scientific applications. WS-PGRADE is a gateway framework that allows users to create such applications by defining them as scientific workflows. This paper investigates how workflow systems and science gateways, such as WS-PGRADE, can be extended with data processing capabilities of Hadoop based on the MapReduce paradigm in the cloud. Analysis shows the methods described to integrate Hadoop with workflows and science gateways work well in different scenarios and can be used to create massively parallel applications for scientific analysis of Big Data.

Keywords

Big Data; cloud computing; data handling; parallel processing; Big Data; Hadoop; MapReduce; WS-PGRADE; cloud computing; gateway framework; parallel computing; scientific workflow system; Big data; Cloud computing; Clouds; Electronic mail; Logic gates; Portals; Servers; cloud computing; hadoop; science gateways; workflow systems;

fLanguage

English

Publisher

ieee

Conference_Titel

Science Gateways (IWSG), 2015 7th International Workshop on

Conference_Location

Budapest

Type

conf

DOI

10.1109/IWSG.2015.15

Filename

7217923