• 系统软件与软件工程 •

1. 1. 华东理工大学 计算机科学与工程系，上海 200237
2. 上海市计算机软件评测重点实验室，上海 201112
3. 上海应用技术大学 计算机科学与信息工程系，上海 200235
• 出版日期:2018-11-01 发布日期:2018-11-12

SUN Huaiying, YU Huiqun, FAN Guisheng, CHEN Liqiong

1. 1. Department of Computer Science and Engineering, East China University of Science and Technology, Shanghai 200237, China
2. Shanghai Key Laboratory of Computer Software Evaluating and Testing, Shanghai 201112, China
3. Department of Computer Science and Information Engineering, Shanghai Institute of Technology, Shanghai 200235, China
• Online:2018-11-01 Published:2018-11-12

Abstract:

Software defined network (SDN) is a network architecture which can separate out the network control functions in the infrastructures and centrally deploy them into a controller. In real-word Hadoop system, there is an NP-complete problem of minimizing the job completion time. This paper combines Hadoop with SDN. With the network control ability of SDN, the available residual bandwidth of the network can be gained as an significant parameter of task scheduling. According to this, a task scheduling algorithm RBA (residual bandwidth based algorithm) is proposed, which can get the approximate optimal allocation schemes for tasks in a job, achieving the goal of minimizing the job completion time. Several simulation experiments are conducted to verify the performance of RBA in terms of job completion time, task data locality and computation time. Experimental results show that RBA is generally better than HDS, BAR, BASS algorithms.