关键词: 云计算, 并行算法, 图数据分析, 数据挖掘, 社会网络分析

Abstract: This paper proposes a parallel data analysis system based on the cloud computing platform of Spark. This system mainly aims at large-scale graph data analysis tasks, supports analysis applications of non-graph data, and integrates the sets of data analysis algorithms and non-graph data analysis algorithms. Then, this paper describes the design and implementation of the system, as well as workflow engine and dynamic component update technology, part of the parallel data analysis algorithms. Through tests of multiple scales of datasets and performance comparison with traditional MapReduce platform, this paper proves that the system is more efficient at completing computing tasks compared with the previous graph data mining system, and can analyze efficiently non-graph data.

Key words: cloud computing, parallel algorithms, graph data analysis, data mining, social network analysis