An Approach to Handle Overload in Real-Time Data Stream Management System

Load shedding is a challengeable issue in data stream management systems (DSMSs). When data stream rates exceed system capacity, the overloaded DSMS fails to process all of its input data and keep up with the rate of data arrival. Especially, in a time-critical environment, queries should be completed not just timely but within certain deadlines. Existing strategies are poor at handling huge fluctuant overload with deadline. In this paper, an Effective Deadline-Aware Random Load Shedding algorithm (named RLS-EDA) is proposed to handle real-time system overload effectively. The RLS-EDA algorithm can make full use of the system idle time by buffering dropped tuples which would have opportunities to be executed when the workload is fade. Experiment results show that our algorithm can reduce average deadline miss ratio and increase system throughput during the period of huge workload fluctuations.