Lifting the Burden of History in Adaptive Ordering of Pipelined Stream Filters

Ordering of commutative and correlated pipelined stream filters in a dynamic environment is a problem of high interest due to its application in data stream scenarios and its relevance to many query optimization problems. Current state-of-the-art adaptive techniques continuously reoptimize filter orderings utilizing statistics that are collected during the execution of the query, however, both the up-to-date and the out-of-date collected statistical data may be considered with the consequence of not always taking effective reoptimization decisions. In this work, we propose a technique for adaptive filter ordering that tries to lift the burden of out-of-date statistical data after detecting changes in the filter drop probabilities. To this end, we propose a novel drop probability change detection algorithm. The experimental evaluation shows that the proposed technique can yield significant performance improvements while decreasing the runtime overhead.