The Stratosphere platform for big data analytics

被引：252

作者：

Alexandrov, Alexander ^{[1
]}

Bergmann, Rico ^{[2
]}

Ewen, Stephan ^{[1
]}

Freytag, Johann-Christoph ^{[2
]}

Hueske, Fabian ^{[1
]}

Heise, Arvid ^{[3
]}

Kao, Odej ^{[1
]}

Leich, Marcus ^{[1
]}

Leser, Ulf ^{[2
]}

Markl, Volker ^{[1
]}

Naumann, Felix ^{[3
]}

Peters, Mathias ^{[2
]}

Rheinlaender, Astrid ^{[2
]}

Sax, Matthias J. ^{[2
]}

Schelter, Sebastian ^{[1
]}

Hoeger, Mareike ^{[1
]}

Tzoumas, Kostas ^{[1
]}

Warneke, Daniel ^{[4
]}

机构：

[1] Tech Univ Berlin, Berlin, Germany

[2] Humboldt Univ, D-10099 Berlin, Germany

[3] Hasso Plattner Inst, Potsdam, Germany

[4] Int Comp Sci Inst, Berkeley, CA 94704 USA

来源：

VLDB JOURNAL | 2014年 / 23卷 / 06期

关键词：

Big data; Parallel databases; Query processing; Query Optimization; Data cleansing; Text mining; Graph processing; Distributed systems;

D O I：

10.1007/s00778-014-0357-y

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

We present Stratosphere, an open-source software stack for parallel data analysis. Stratosphere brings together a unique set of features that allow the expressive, easy, and efficient programming of analytical applications at very large scale. Stratosphere's features include "in situ" data processing, a declarative query language, treatment of user-defined functions as first-class citizens, automatic program parallelization and optimization, support for iterative programs, and a scalable and efficient execution engine. Stratosphere covers a variety of "Big Data" use cases, such as data warehousing, information extraction and integration, data cleansing, graph analysis, and statistical analysis applications. In this paper, we present the overall system architecture design decisions, introduce Stratosphere through example queries, and then dive into the internal workings of the system's components that relate to extensibility, programming model, optimization, and query execution. We experimentally compare Stratosphere against popular open-source alternatives, and we conclude with a research outlook for the next years.

引用

页码：939 / 964

页数：26

共 64 条

[1] Ackermann S., 2012, BIG DAT WORKSH VLDB
[2] Massively Parallel Data Analysis with PACTs on Nephele
Alexandrov, Alexander
Heimel, Max
Markl, Volker
Battre, Dominic
Hueske, Fabian
Nijkamp, Erik
Ewen, Stephan
Kao, Odej
Warneke, Daniel
[J]. PROCEEDINGS OF THE VLDB ENDOWMENT, 2010, 3 (02): : 1625 - 1628
[3] Alexandrov Alexander., 2011, BTW, P25
[4] [Anonymous], 2005, Scientific Programming
[5] [Anonymous], 2010, ICWSM
[6] [Anonymous], 2012, P 9 USENIX C NET WOR
[7] [Anonymous], 2013, CIDR
[8] [Anonymous], P 2010 ACM SIGMOD IN, DOI [DOI 10.1145/1807167.1807184, 10.1145/1807167.1807184]
[9] [Anonymous], 2003, P 19 ACM S OP SYST P, DOI [10.1145/1165389.945450, DOI 10.1145/1165389.945450]
[10] [Anonymous], 2010, SoCC, DOI DOI 10.1145/1807128.1807148

← 1 2 3 4 5 6 7 →