Hadoop-GIS: A High Performance Spatial Data Warehousing System over MapReduce

被引：356

作者：

Aji, Ablimit ^{[1
]}

Wang, Fusheng ^{[2
]}

Vo, Hoang ^{[1
]}

Lee, Rubao ^{[3
]}

Liu, Qiaoling ^{[1
]}

Zhang, Xiaodong ^{[3
]}

Saltz, Joel ^{[2
]}

机构：

[1] Emory Univ, Dept Math & Comp Sci, Atlanta, GA 30322 USA

[2] Emory Univ, Dept Biomed Informat, Atlanta, GA 30322 USA

[3] Ohio State Univ, Dept Comp Sci & Engn, Columbus, OH 43210 USA

来源：

PROCEEDINGS OF THE VLDB ENDOWMENT | 2013年 / 6卷 / 11期

关键词：

D O I：

10.14778/2536222.2536227

中图分类号：

TP [自动化技术、计算机技术];

学科分类号：

0812 ;

摘要：

Support of high performance queries on large volumes of spatial data becomes increasingly important in many application domains, including geospatial problems in numerous fields, location based services, and emerging scientific applications that are increasingly data- and compute-intensive. The emergence of massive scale spatial data is due to the proliferation of cost effective and ubiquitous positioning technologies, development of high resolution imaging technologies, and contribution from a large number of community users. There are two major challenges for managing and querying massive spatial data to support spatial queries: the explosion of spatial data, and the high computational complexity of spatial queries. In this paper, we present Hadoop-GIS a scalable and high performance spatial data warehousing system for running large scale spatial queries on Hadoop. Hadoop-GIS supports multiple types of spatial queries on MapReduce through spatial partitioning, customizable spatial query engine RESQUE, implicit parallel spatial query execution on MapReduce, and effective methods for amending query results through handling boundary objects. Hadoop-GIS utilizes global partition indexing and customizable on demand local spatial indexing to achieve efficient query processing. Hadoop-GIS is integrated into Hive to support declarative spatial queries with an integrated architecture. Our experiments have demonstrated the high efficiency of Hadoop-GIS on query response and high scalability to run on commodity clusters. Our comparative experiments have showed that performance of Hadoop-GIS is on par with parallel SDBMS and outperforms SDBMS for compute-intensive queries. Hadoop-GIS is available as a set of library for processing spatial queries, and as an integrated software package in Hive.

引用

页码：1009 / 1020

页数：12

共 32 条

[1]

Abouzeid A, 2009, P VLDB, V2, P922

[2]

Aji Ablimit, 2012, Proc ACM SIGSPATIAL Int Conf Adv Inf, V2012, P309

[3]

Aji Ablimit, 2012, P SIGMOD PODS 2012 P, P9

[4]

Akdogan A., 2010, Proceedings of the 2010 IEEE 2nd International Conference on Cloud Computing Technology and Science (CloudCom 2010), P9, DOI 10.1109/CloudCom.2010.92

[5]

BECKMANN N, 1990, SIGMOD

[6]

Blanas S., 2010, SIGMOD

[7]

Brinkhoff Thomas, 1996, ICDE

[8]

Cary A, 2009, LECT NOTES COMPUT SC, V5566, P302, DOI 10.1007/978-3-642-02279-1_24

[9] SCOPE: Easy and Efficient Parallel Processing of Massive Data Sets [J].

Chaiken, Ronnie ;

Jenkins, Bob ;

Larson, Per-Ake ;

Ramsey, Bill ;

Shakib, Darren ;

Weaver, Simon ;

Zhou, Jingren .

PROCEEDINGS OF THE VLDB ENDOWMENT, 2008, 1 (02) :1265-1276

[10] MapReduce: A Flexible Data Processing Tool [J].

Dean, Jeffrey ;

Ghemawat, Sanjay .

COMMUNICATIONS OF THE ACM, 2010, 53 (01) :72-77

← 1 2 3 4 →