Programming Abstraction for Resource Aware Stream Processing for Scientific Workflows

As the volume of real time data available for use in scientific discovery explodes, the limiting factor is increasingly the amount of time and attention a scientist can give to a problem. Processing event streams from heterogeneous sources in real time adds a dimension to e-Science workflow systems that is less well understood. Considering the types of computations that scientific workflows focus on and the latencies associated with them, it is not immediately evident that scientific workflows can directly apply to high throughput real time event processing. In this paper we propose a model for extending an established scientific workflow system to incorporate event processing without losing the richness of the programming abstraction.