SDPPF — A MapReduce based parallel processing framework for spatial data

Author

Zhao, Dong ; Huang, Zhen-Chun

Author_Institution

Dept. of Comput. Sci. & Technol., Tsinghua Nat. Lab. for Inf. Sci. & Technol. (TNLIST), Beijing, China

fYear

2011

fDate

16-18 Sept. 2011

Firstpage

1258

Lastpage

1261

Abstract

Spatial data processing requires large scale calculation and parallel processing is necessary for accelerating. Popular parallel frameworks need lots of code development for parallelization, which are difficult for geo-scientists who are often beginners in programming. In this paper, an easy-to-use parallel processing framework is proposed and named SDPPF Spatial Data Parallel Processing Framework. SDPPF is MapReduce based and can directly reuse existing binary executable program for parallel processing, especially when the source code of specific algorithm is not able to get. The parallel model, architecture and implementation of SDPPF are presented and the evaluation of SDPPF is analyzed by testing specific algorithms. Experiments show that SDPPF is a flexible, easy-to-use and scalable framework for spatial data parallel processing.

Keywords

geophysics computing; parallel processing; program compilers; software reusability; source coding; visual databases; MapReduce based parallel processing framework; SDPPF; binary executable program reusability; code development; easy-to-use parallel processing framework; geoscientists; source code; spatial data parallel processing framework; Computational modeling; Data processing; Processor scheduling; Programming; Runtime environment; Spatial databases; parallel processing framework; spatial data;

fLanguage

English

Publisher

ieee

Conference_Titel

Electrical and Control Engineering (ICECE), 2011 International Conference on

Conference_Location

Yichang

Print_ISBN

978-1-4244-8162-0

Type

conf

DOI

10.1109/ICECENG.2011.6057775

Filename

6057775