M2M:A Simple Matlab-to-MapReduce Translator for Cloud Computing

被引：2

作者：

Junbo Zhang ^{[1
,2
]}

Dong Xiang ^{[3
]}

Tianrui Li ^{[4
]}

Yi Pan ^{[5
]}

机构：

[1] the School of Information Scienceand Technology,Southwest Jiaotong University

[2] Department of Computer Science,Georgia State University

[3] the School of Information Science and Technology,Southwest Jiaotong University

[4] the School of Software,Tsinghua University

[5] the Department of Computer Science,Georgia State University

来源：

TsinghuaScienceandTechnology | 2013年 / 18卷 / 01期

基金：

中国国家自然科学基金; 美国国家科学基金会;

关键词：

MapReduce; Matlab; translator; cloud computing;

D O I：

暂无

中图分类号：

TP3 [计算技术、计算机技术];

学科分类号：

0812 ;

摘要：

MapReduce is a very popular parallel programming model for cloud computing platforms, and has become an effective method for processing massive data by using a cluster of computers. X-to-MapReduce (X is a program language) translator is a possible solution to help traditional programmers easily deploy an application to cloud systems through translating sequential codes to MapReduce codes. Recently, some SQL-to-MapReduce translators emerge to translate SQL-like queries to MapReduce codes and have good performance in cloud systems. However, SQL-to-MapReduce translators mainly focus on SQL-like queries, but not on numerical computation. Matlab is a high-level language and interactive environment for numerical computation, visualization, and programming, which is very popular in engineering. We propose and develop a simple Matlab-to-MapReduce translator for cloud computing, called M2M, for basic numerical computations. M2M can translate a Matlab code with up to 100 commands to MapReduce code in few seconds, which may cost a proficient Hadoop MapReduce programmer some days on coding so many commands. In addition, M2M can also recognize the dependency between complex commands, which is always confusing during hand coding. We implemented M2M with evaluation for Matlab commands on a cluster. Several common commands are used in our experiments. The results show that M2M is comparable in performance with hand-coded programs.

引用

页码：1 / 9

页数：9

共 16 条

[1]

HadoopOnAzure. https://www.hadooponazure.com/ . 2012

[2]

Mahout in action. Sean Owen,Robin Anil,Ted Dunning. . 2011

[3]

Performance-driven task co-scheduling for mapreduce environments. J.Polo,D.Carrera,Y.Becerra,J.Torres,E.Ayguade′,M.Steinder,I.Whalley. Network Operations and Management Symposium(NOMS),2010IEEE . 2010

[4]

Parallel programming on cloud computing platforms:Challenges and solutions. Y.Pan,J.Zhang. KITCS/FTRA Journal of Convergence . 2012

[5]

Ysmart:Yet another sql-to-mapreduce translator. R.Lee,T.Luo,Y.Huai,F.Wang,Y.He,X.Zhang. Distributed Computing Systems(ICDCS),201131st Int Conf.on . 2011

[6]

MATLAB:An Introduction with Applications. A.Gilat. . 2011

[7]

Amazon Elastic MapReduce. http://aws.amazon.com/elasticmapreduce/ . 2012

[8]

Mapreduce:Simplified data processing on large clusters. J.Dean,S.Ghemawat. Communications of the ACM . 2008

[9]

Mars:A mapreduce framework on graphics processors. B.He,W.Fang,Q.Luo,N.K.Govindaraju,T.Wang. Proc of the17th International Conference on Parallel Architectures and Compilation Techniques . 2008

[10]

Portable parallel programming on cloud and hpc:Scientific applications of twister4azure. T.Gunarathne,B.Zhang,T.-L.Wu,,J.Qiu. Utility and Cloud Computing(UCC),2011Fourth IEEE Int Conf.on . 2011

← 1 2 →