Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (9): 1990-2010.DOI: 10.3778/j.issn.1673-9418.2202063

• Surveys and Frontiers • Previous Articles     Next Articles

Review of Image Super-resolution Reconstruction Algorithms Based on Deep Learning

YANG Caidong1, LI Chengyang1,2, LI Zhongbo1,+(), XIE Yongqiang1, SUN Fangwei1, QI Jin1   

  1. 1. Institute of Systems Engineering, Academy of Military Sciences, Beijing 100141, China
    2. School of Electronics Engineering and Computer Science, Peking University, Beijing 100871, China
  • Received:2022-02-23 Revised:2022-05-26 Online:2022-09-01 Published:2022-09-15
  • About author:YANG Caidong, born in 1996, M.S. candidate. His research interests include super-resolution reconstruction and object detection.
    LI Chengyang, born in 1995, Ph.D. candidate. His research interests include object detection, data mining and machine learning.
    LI Zhongbo, born in 1983, Ph.D., senior engineer. His research interests include multimedia technology, machine learning, cloud computing, etc.
    XIE Yongqiang, born in 1972, Ph.D., researcher. His research interests include machine learning, multimedia technology, intelligent system architecture, etc.
    SUN Fangwei, born in 1996, M.S. candidate. His research interests include object detection, object tracking and semantic segmentation.
    QI Jin, born in 1971, M.S., senior engineer. Her research interests include multimedia technology, intelligent system architecture, cloud computing, etc.

深度学习的图像超分辨率重建技术综述

杨才东1, 李承阳1,2, 李忠博1,+(), 谢永强1, 孙方伟1, 齐锦1   

  1. 1.军事科学院 系统工程研究院,北京 100141
    2.北京大学 信息科学与技术学院,北京 100871
  • 通讯作者: + E-mail: zbli2021@163.com
  • 作者简介:杨才东(1996—),男,贵州六盘水人,硕士研究生,主要研究方向为超分辨率重建、目标检测。
    李承阳(1995—),男,辽宁鞍山人,博士研究生,主要研究方向为目标检测、数据挖掘、机器学习。
    李忠博(1983—),男,北京人,博士,高级工程师,主要研究方向为多媒体技术、机器学习、云计算等。
    谢永强(1972—),男,北京人,博士,研究员,主要研究方向为机器学习、多媒体技术、智能系统架构等。
    孙方伟(1996—),男,山东青岛人,硕士研究生,主要研究方向为目标检测、目标跟踪、语义分割。
    齐锦(1971—),女,北京人,硕士,高级工程师,主要研究方向为多媒体技术、智能系统架构、云计算等。

Abstract:

The essence of image super-resolution reconstruction technology is to break through the limitation of hardware conditions, and reconstruct a high-resolution image from a low-resolution image which contains less infor-mation through the image super-resolution reconstruction algorithms. With the development of the technology on deep learning, deep learning has been introduced into the image super-resolution reconstruction field. This paper summarizes the image super-resolution reconstruction algorithms based on deep learning, classifies, analyzes and compares the typical algorithms. Firstly, the model framework, upsampling method, nonlinear mapping learning module and loss function of single image super-resolution reconstruction method are introduced in detail. Secondly, the reference-based super-resolution reconstruction method is analyzed from two aspects: pixel alignment and Patch matching. Then, the benchmark datasets and image quality evaluation indices used for image super-resolution recon-struction algorithms are summarized, the characteristics and performance of the typical super-resolution recons-truction algorithms are compared and analyzed. Finally, the future research trend on the image super-resolution reconstruction algorithms based on deep learning is prospected.

Key words: super-resolution reconstruction, deep learning, single image, reference-based, image alignment

摘要:

图像超分辨率重建技术的本质是突破现有硬件条件的限制,通过算法将低分辨率图像重建为高分辨率图像,获得包含更多信息的图像的技术。随着深度学习理论和技术的迅速发展,深度学习被引入到超分辨率重建领域并取得了进展。对基于深度学习的图像超分辨率重建算法进行了全面总结,并对已有算法进行了分类、分析和比较。首先,详细介绍了单图像超分辨率重建模型的组成结构,包括超分框架、上采样方法、非线性映射学习模块以及损失函数等。其次,从图像对齐和Patch匹配两方面出发,对现有的基于参考的图像超分辨率重建算法进行了分析。然后,介绍了图像超分辨重建领域的benchmark数据集以及图像质量评估参数,对目前主流算法的性能进行了评估。最后,对基于深度学习的图像超分辨率重建算法的未来研究趋势进行了展望。

关键词: 超分辨率重建, 深度学习, 单图像, 基于参考, 图像对齐

CLC Number: