Utility-aware Resource Allocation in an Event Processing System

Event processing systems link event producers and consumers in a flexible manner, by supporting multiple communication patterns and powerful message transformations. Such systems support both loosely coupled communication patterns, such as one-to-one or one-to-many and tightly coupled (request-response) patterns. An important concern in these systems is the allocation of resources used to support event processing. These include traditional constrained resources such as CPU and network, as well as an increasing reliance on storage resources. For instance, transactional and reliable producers require events to be logged to stable storage. In this paper we consider the problem of utility driven allocation of these resources, taking into account resource capacity. In contrast to previous work, we (1) develop a unified utility model to deal with throughput requirements of one-way flows and latency requirements of request-response flows, (2) differentiate between classes of consumers for one-to-many flows and subject these classes to consumer admission control, (3) compute system optimization constraints using both resource bandwidth and latency and (4) develop a control architecture that combines periodic optimization with continuous rate control, both of which are utility aware. We describe an experimental evaluation of our approach on a prototype event processing server.