Flexible IO services in the Kepler grid workflow system

Existing grid workflow tools assume that individual components either communicate by passing files from one application to another, or are explicitly linked using interprocess communication pipes. When files are used it is usually difficult to overlap reader and write execution. On the other hand, interprocess communication primitives are invasive and require substantial modification to existing code. We have built a library, called GriddLeS, that combines the advantages of both approaches without any changes to the application code. GriddLeS overloads conventional file IO operations, and supports either local, remote or replicated file access, or direct communication over pipes. In this paper we discuss how GriddLeS can be combined with workflow packages and show how flexible and powerful virtual applications can be constructed rapidly. A large atmospheric science case study is discussed