计算机科学与探索 ›› 2017, Vol. 11 ›› Issue (7): 1021-1032.DOI: 10.3778/j.issn.1673-9418.1702022

• 综述·探索 • 上一篇    下一篇

性能收益模型的流处理算子优化技术综述

檀国林1,2,3,海  玲4+,张  鹏1,2,3,陈志鹏1,2,3   

  1. 1. 中国科学院 信息工程研究所,北京 100093
    2. 信息内容安全技术国家工程实验室,北京 100093
    3. 中国科学院大学,北京 100049
    4. 新疆工程学院,乌鲁木齐 830091
  • 出版日期:2017-07-01 发布日期:2017-07-07

Survey of Stream Processing Operator Optimizations for Performance Gain

TAN Guolin1,2,3, HAI Ling4+, ZHANG Peng1,2,3, CHEN Zhipeng1,2,3   

  1. 1. Institute of Information Engineering, Chinese Academy of Sciences, Beijing 100093, China
    2. National Engineering Laboratory for Information Security Technologies, Beijing 100093, China
    3. University of Chinese Academy of Sciences, Beijing 100049, China
    4. Xinjiang Institute of Engineering, Urumchi 830091, China
  • Online:2017-07-01 Published:2017-07-07

摘要: 大数据移动互联网时代的到来,数据量也越来越庞大,数据之大使得对数据进行高效实时处理的需求也变得越来越迫切,促使国内外的研究团队开发出许多流处理应用。为了提高流处理应用的性能,这些流处理应用底层实现都采用了各种各样复杂的流处理算子优化技术。在调研学习这些流处理应用的基础上,概括总结了其中最常见的8种流处理算子优化技术,并结合实际例子,分别从性能收益、安全条件、动态性等方面详细介绍了这些算子优化技术的特点,并探讨了算子优化和流处理应用领域进一步的研究方向。

关键词: 算子优化, 流处理, 性能收益

Abstract: The arrival of the era of big data and mobile Internet lets people be in data torrent. Big data make the need more and more urgent to process data efficiently and real-timely. And big data also prompt research teams around the world to develop a lot of stream processing applications. The implementations of these stream processing applications use a variety of operator optimizations. Based on the research on these stream processing applications, this paper summarizes the most common eight operator optimizations of stream processing applications. And combined with practical examples, this paper introduces the features of these operator optimizations from aspects of performance gain, safety condition and dynamic. Then this paper discusses the further research direction in the field of operator optimizations and stream processing.

Key words: operator optimization, stream processing, performance gain