Journal of Frontiers of Computer Science and Technology ›› 2013, Vol. 7 ›› Issue (10): 865-885.DOI: 10.3778/j.issn.1673-9418.1307035

Previous Articles     Next Articles

A Survey on MapReduce Optimization Technologies

东北大学 信息科学与工程学院,沈阳 110819   

  1. College of Information Science and Engineering, Northeastern University, Shenyang 110819, China
  • Online:2013-10-01 Published:2013-09-30

MapReduce优化技术综述

黄  山,王波涛+,王国仁,于  戈,李佳佳   

  1. HUANG Shan, WANG Botao+, WANG Guoren, YU Ge, LI Jiajia

Abstract: As a parallel programming model for big data processing, MapReduce is getting more and more attractions from academia and industry for its good scalability, availability and fault tolerance. There exist a lot of optimization technologies focusing on the application limitations of MapReduce. This paper firstly introduces the MapReduce framework, then compares the research work on MapReduce optimization technologies including column storage, index, join, iteration calculation, scientific calculation, and scheduling algorithms respectively. Finally, this paper analyzes the challenges and figures out the trends of this area.

Key words: MapReduce, column storage, index, join, iteration, scientific calculation, scheduling algorithms, optimization

摘要: 作为一种处理大数据的并行编程模型,MapReduce由于其良好的可扩展性、可用性、容错性,得到了学术界和工业界的关注。针对MapReduce在应用领域中的不足,已经存在大量的优化技术。介绍了MapReduce框架,比较了现存的MapReduce列存储、索引、连接、迭代计算、科学计算及调度算法方面的优化技术,分析了MapReduce技术研究的挑战性问题,指出了未来研究方向。

关键词: MapReduce, 列存储, 索引, 连接, 迭代, 科学计算, 调度算法, 优化