Just-In-Time Processing of Continuous Queries

In a data stream management system, a continuous query is processed by an execution plan consisting of multiple operators connected via the "consumer-producer" relationship, i.e., the output of an operator (the "producer") feeds to another downstream operator (the "consumer") as input. Existing techniques execute each operator separately and push all results to its consumers, without considering whether the consumers need them. Consequently, considerable CPU and memory resources are wasted on producing and storing useless intermediate results. Motivated by this, we propose just-in-time (JIT) processing, a novel methodology that enables a consumer to return feedback expressing its current demand to the producer. The latter selectively generates results based on this information. We show, through extensive experiments, that JIT achieves significant savings in terms of both CPU time and memory consumption.