Data partitioning over data streams based on change-aware sampling

Author

Wang, Yongli ; Xu, Hongbing ; Dong, Yisheng ; Liu, Xuejun ; Qian, Jiangbo

Author_Institution

Dept. of Comput. Sci. & Eng., Southeast Univ., Nanjing

fYear

2005

fDate

12-18 Oct. 2005

Firstpage

582

Lastpage

585

Abstract

A novel data partitioning method adapted to a distributed parallel streams processing system for power industry is proposed. This method uses change-aware sampling algorithm that can guarantee low error to describe the distribution characteristics of the data-values first. And then it uses an improved heuristic constructing equal depth histograms algorithm to generate approximate partition vector efficiently. Experiments results on actual data prove that the proposed method is efficient, practical and suitable for time-varying data streams processing

Keywords

data analysis; electricity supply industry; parallel databases; power engineering computing; sampling methods; temporal databases; change-aware sampling algorithm; data partitioning; distributed parallel stream processing system; equal depth histogram algorithm; power industry; time-varying data stream processing; Computer science; Concurrent computing; Data engineering; Distributed computing; Frequency; Histograms; Partitioning algorithms; Power engineering and energy; Reservoirs; Sampling methods;

fLanguage

English

Publisher

ieee

Conference_Titel

e-Business Engineering, 2005. ICEBE 2005. IEEE International Conference on

Conference_Location

Beijing

Print_ISBN

0-7695-2430-3

Type

conf

DOI

10.1109/ICEBE.2005.47

Filename

1552950