ارائه روشي پويا جهت پاسخ به پرس‌ و جوهاي پيوسته تجمّعي اقتضايي

Providing a Dynamic Technique for Answering Ad-hoc Continuous Aggregate

جريان‌هاي داده دنباله‌هاي نامتناهي، سريع، متغير با زمان و با نرخ ورود انفجاري از نشان‌هاي داده هستند كه به‌ طور معمول نياز دارند به‌صورت برخط و به‌طورتقريبي بي‌درنگ پردازش شوند. بر اين اساس، الگوريتم‌هاي پردازش جريان‌هاي داده و اجراي پرس‌ و جوها روي جريان داده‌ها بيش‌تر تك‌گذره هستند. اجراي اين الگوريتم‌هاي تك‌گذره با محدوديت‌ها و چالش‌هايي از قبيل محدوديت در حافظه، زمان‌بندي، و دقت پاسخ‌ها مواجه است. اين چالش‌ها به‌ويژه در شرايطي كه پرس‌وجوي مورد نظر از قبل تعيين و مشخص نشده باشد و به‌صورت اقتضايي، پس از ارسال جريان داده ارائه شود، به‌مراتب جد‌ي‌تر و حل آن‌ها دشوارتر خواهد بود. در اين مقاله، براي پردازش پرس‌وجوهاي تجمعي كه به‌طور پيوسته روي جريان‌هاي داده اجرا خواهند شد و البته به‌طور اقتضايي ارائه مي‌شوند، راه حلي مبتني بر ساختار درختواره و نگهداشت نتايج تجمعي معرفي شده است. نكته مهم در اين روش، برقراري برخط بودن در تمام مراحل ساخت، نگهداري و بهره‌برداري از درخت است. براي تأمين برخط بودن فرايند پاسخ به پرس‌وجو، كافي است تمامي پاسخ‌هاي محتمل را نگهداري كنيم؛ اما براي حفظ برخط‌بودن فرايند ساخت و نگهداري درخت، با توجه به ويژگي‌هاي ذاتي جريان داده ناچاريم برخي پاسخ‌ها را نگهداري كنيم. بدين ترتيب، هدف و مسئله اساسي آن است كه دست‌كم پاسخ‌هاي انتخابي براي ذخيره‌ در قالب درختواره را به مجموعه پاسخ‌هاي مورد نياز براي پرس‌ و جوهاي اقتضايي رسيده نزديك‌تر كنيم. ساختار درخت تجمعي پيشوندي پيشنهادي كه به‌صورت پويا ايجاد، نگهداري، مديريت و در پردازش پرس‌وجوها استفاده مي‌شود، تشريح و صحت عملكرد آن به‌صورت عملي مورد ارزيابي قرار گرفته كه نتايج حاكي از كارآمد‌بودن آن براي به‌كارگيري در پردازش برخط پرس‌ و جوهاي پيوسته تجمعي اقتضايي روي جريان‌هاي داده است.

Data Streams are infinite, fast, time-stamp data elements which are received explosively. Generally, these elements need to be processed in an online, real-time way. So, algorithms to process data streams and answer queries on these streams are mostly one-pass. The execution of such algorithms has some challenges such as memory limitation, scheduling, and accuracy of answers. They will be more important and serious, chiefly if the queries are not predefined but Ad-hoc, and also should be executed after data stream tuples are gone. Countinous aggregate queries are types of queries with some special characteristics making it possible to perform more specific, efficient qeury processing techniques, specifiaclly beneficient for ad-hoc ones. In this paper, a dynamic efficient techinque is proposed for answering the ad-hoc continiues aggregate queries over data streams. The main idea of the proposed technique is to generate and handle an efficiet tree data structure as the synopse, in the form of Dynamic Prefix Aggregate Tree. In general, the two following approaches can be used to calculate any function such as ; either implementation of an algorithm for the calculation of function f, or storing the answers of function f for all possible states. When the algorithm runtime is high, the second method strengthened by proper selection of indices can return a proper answer in a very short time (even ). But the major problem of the second method is the total number of possible answers which can be very high and also can be out of the possible storage capacity and processing potential within a certain acceptable time period. For example, suppose that the cardinality of each of the parameters of is 10. In this case, the total number of possible states will be . As it is evident, the total number of states increases with the number of parameters and their cardinalities.When the total number of states is so great that generating answers with respect to consumed time and space is impossible, a more convenient, practical method should be employed. This more practical approach can be the storing of some of the answers (selectively) with respect to the following conditions: Obtaining un-stored answers from the set of stored answers. Higher probability of utilizing stored answers (i.e. higher probability of submitting requests from stored set). Eliminating (not storing) null answers. The same idea can be implemented for online and almost real time processing of queries, so that by receiving each tuple, all possible answers get obtained and stored. By doing so, in the time of need (when answering to an ad-hoc query) stored answers will be used instead of calculating each answer. Accordingly, some answers are stored in a tree structure to be used at the right time. In this paper, in order to answer ad-hoc continuous aggregate queries over data streams, a method is proposed that uses a tree structure for storing the aggregate results. The important point in this method is that all steps of the construction, maintenance and using of the tree must be online. For these purposes, it is enough to keep all possible answers. But to apply an online construction and maintenance of tree, we must keep some answers, according to the inherent features of data streams. In this way, the main goal is to choose the answers possessing the most overlap with responses answers of received ad-hoc queries. The proposed method, creates the tree structure and maintains it dynamically to answer ad-hoc aggregate continuous queries over data streams. For this purpose, queries at instant are modeled as in form of , where or (when , the aggregate over the whole sliding window is returned) and is the size of sliding window and (when , the aggregate over the whole is returned). In order to increase the overlapping, a statistical task is performed on a dimensions of the received queries. In this way, dimensions are determined with the highest, lowest request. When , means that there is no request for this dimension. Therefore, we select and store the answers related to the dimension with highest request, and ignore those with the lowest. Obviously, these answers should be obtained and presented using stored answers. As the request for dimensions may change, the tree structure must be dynamically constructed and maintenance that will be presented this dynamic structure in this paper. Experimental evaluattion of the proposed method shows that, using the proposed Dynamic Aggregate Tree for ansering countinous Ad-hoc aggregate queies is more cost-effective, in terms of response time and memory usage.