Why Not Semijoins for Streams, When Distributed?

This paper addresses the semijoin-based window join algorithm over distributed data streams. In distributed stream query processing, data streams arriving at remote sites need to be shipped to the processing site for query execution. This typically introduces high communication overhead over the network. Our observation is that semi- join, effective to reduce communication overhead in distributed database query processing, can be also effective in distributed stream query processing. The challenge, of course, lies in the streaming nature of tuples, the processing of which is fundamentally different from processing a set of tuples. We address this challenge by first adapting the window-based stream join to a distributed environment. The resulting join algorithm (called simple join) uses the idea of exporting a window to the query processing site. We then adopt the semijoin to reduce the communication overhead (in return for a marginal increase of the processing overhead). The resulting semijoin-based join algorithm uses the ideas of a mirror window and a partial tuple. That is, it creates a copy of a remote window at the processing site and sends a partial tuple to probe for matching tuples before sending a full tuple. Finally, we analyze the two join algorithms using our proposed cost models and verify the analysis results through a set of experiments.