关键词: 云计算, 中间数据, 副本, 容错算法

Abstract: Cloud computing frameworks usually generate large amounts of intermediate data which are short-lived, yet are important for the completion of job. Once there are server failures, it will lead to the failures of intermediate data, and affects the computation of the whole job. However, the existing fault-tolerant processing approaches only adopt simple replication strategies which can incur significant network overhead, and have no considering of the characteristics of intermediate data. Therefore, this paper proposes an efficient layered intermediate data fault-tolerant approach, named IDF_Support (intermediate data fault-tolerance_support) approach. By dividing the computing tasks into different classifications, IDF_Support approach can effectively process the intermediate data failures. Then, this paper proposes two layered intermediate data fault-tolerant algorithms, respectively the inner task intermediate data fault-tolerant algorithm (Inner_Task IDF) which resolves the fault-tolerance within a task and the outer task intermediate data fault-tolerant algorithm (Outer_Task IDF) which resolves the fault-tolerance among tasks. The experimental results show that the proposed algorithms can improve the response time in the case of machine failure, and keep the reliability of the whole system.

Key words: cloud computing, intermediate data, replication, fault-tolerant algorithm