关键词: 软件定义网络（SDN）, Hadoop, 带宽, 任务调度

Abstract:

Software defined network (SDN) is a network architecture which can separate out the network control functions in the infrastructures and centrally deploy them into a controller. In real-word Hadoop system, there is an NP-complete problem of minimizing the job completion time. This paper combines Hadoop with SDN. With the network control ability of SDN, the available residual bandwidth of the network can be gained as an significant parameter of task scheduling. According to this, a task scheduling algorithm RBA (residual bandwidth based algorithm) is proposed, which can get the approximate optimal allocation schemes for tasks in a job, achieving the goal of minimizing the job completion time. Several simulation experiments are conducted to verify the performance of RBA in terms of job completion time, task data locality and computation time. Experimental results show that RBA is generally better than HDS, BAR, BASS algorithms.

Key words: software defined network (SDN), Hadoop, bandwidth, task scheduling