Efficient processing of semi-stream data

Semi-stream processing has become an emerging area of research in the field of data stream management. One common operation in semi-stream processing is joining a stream with disk-based data using a join operator. This join operator typically works under limited main memory and this memory is generally not large enough to hold the whole disk-based data. Recently, a number of semi-stream join algorithms have been proposed in the literature to achieve an optimal performance but still there is room to improve the performance. In this invited paper we propose a novel Semi-Stream Join using a new cache module. The algorithm is more appropriate for skewed distributions, and particularly we consider Zipfian distributions of the type that appears in many applications. For such distributions the new algorithm significantly outperforms the existing approaches.