Curracurrong cloud: Stream processing in the cloud

The dominant model for computing with large-scale data in cloud environments has been founded on batch processing including the Map-Reduce model. Important use-cases such as monitoring and alerting in the cloud require instead the incremental and continual handling of new data. Thus recent systems such as Storm, Samza and S4 have adopted ideas from stream processing to the cloud environment. We describe a novel system, Curracurrong Cloud, that, for the first time, allows the computation and data origins to share a cloud-hosted cluster, offers a lightweight algebraic-style description of the processing pipeline, and supports automated placement of computation among compute resources.