DocumentCode :
1689789
Title :
The evolution of a hierarchical partitioning algorithm for large-scale scientific data: three steps of increasing complexity
Author :
Baldwin, Chuck ; Eliassi-Rad, Tina ; Abdulla, Ghaleb ; Critchlow, Terence
Author_Institution :
Lawrence Livermore Nat. Libr., UK
fYear :
2003
Firstpage :
225
Lastpage :
228
Abstract :
As scientific data sets grow exponentially in size, the need for scalable algorithms that heuristically partition the data increases. In this paper, we describe the three-step evolution of a hierarchical partitioning algorithm for large-scale spatio-temporal scientific data sets generated by massive simulations. The first version of our algorithm uses a simple top-down partitioning technique, which divides the data by using a four-way bisection of the spatio-temporal space. The shortcomings of this algorithm lead to the second version of our partitioning algorithm, which uses a bottom-up approach. In this version, a partition hierarchy is constructed by systematically agglomerating the underlying Cartesian grid that is placed on the data. Finally, the third version of our algorithm utilizes the intrinsic topology of the data given in the original scientific problem to build the partition hierarchy in a bottom-up fashion. Specifically, the topology is used to heuristically agglomerate the data at each level of the partition hierarchy. Despite the growing complexity in our algorithms, the third version of our algorithm builds partition hierarchies in less time and is able to build trees for larger size data sets as compared to the previous two versions.
Keywords :
data handling; data models; spatial data structures; temporal databases; visual databases; Cartesian grid; bottom-up approach; data partitioning; data topology; hierarchical partitioning; large-scale data set; large-scale scientific data; scientific data set; spatio-temporal data set; top-down partitioning; Astrophysics; Computational modeling; Heuristic algorithms; Iterative algorithms; Laboratories; Large-scale systems; Partitioning algorithms; Spatial databases; Spatiotemporal phenomena; Topology;
fLanguage :
English
Publisher :
ieee
Conference_Titel :
Scientific and Statistical Database Management, 2003. 15th International Conference on
ISSN :
1099-3371
Print_ISBN :
0-7695-1964-4
Type :
conf
DOI :
10.1109/SSDM.2003.1214983
Filename :
1214983
Link To Document :
بازگشت