A Study on a Lightweight Scientific Workflow System for Astronomical E-Science Service

Network-grid computing is widely adopted in complex data computing and distributed data processing. In order to construct loosely-coupled application, the scientific workflow technique was used to invoke distributed services and data. However, there are many drawbacks in current scientific workflow system such as the complexity of manipulation and the difficulty of system deploy. In this paper, aims to the urgent requirement of astronomic data processing, we have a study on a lightweight scientific workflow system and discuss the critical techniques of the system implementation. Finally, we present a lightweight scientific workflow system, which named C-SWF, to support the complicated data processing in the field of astronomy. C-SWF fully implemented the critical features such as task customization, job scheduling, data movement, data´s provenance and task re-run mechanism. Comparing with the existing scientific workflow system, C-SWF has many useful features that simplify the manipulation and deployment of the system.