关键词: MapReduce, 优化算法, 大数据, 排序算法

Abstract: MapReduce has become the standard parallel computing model on big data analysis. To balance highly the loading nodes in MapReduce system and minimize space usage, CPU, I/O operation time and network overhead, based on a good parallel algorithm, this paper proposes an optimization algorithm of MapReduce design specification, optimizing indexes at the same time. This paper also gives theoretical analysis for the most important sorting algorithm in data processing field, presents the optimal algorithm of multiple index constraints, and proves that the optimal algorithm meets the standard of MapReduce optimization algorithm. The experiments verify that this optimal sorting algorithm is better than the traditional sorting algorithm in terms of effectiveness and efficiency.

Key words: MapReduce, optimization algorithm, big data, sorting algorithm