单幅图像去雨的三阶段通道分割密集融合网络

doi:10.3778/j.issn.1673-9418.2401036

摘要/Abstract

摘要： 单幅图像的雨纹去除是图像处理中的一个挑战。一种有效的单幅图像去雨算法可以显著提高恶劣天气条件下的图像质量。现有的去雨算法往往参数较多，并且随着网络层数的不断增加，图像特征的提取出现冗余，严重影响去雨的效果以及背景的细节信息。因此，提出了一种三阶段通道分割密集融合网络（TCSDFNet）以获得更好的去雨性能。在网络中采用多个通道分割模块（CSB）进行密集特征融合，来聚集雨纹的高、低级别特征。CSB采用通道分割操作将雨天图像分割成多个通道，根据不同层次的特征应用不同的雨纹去除方法，减少冗余特征和网络参数，提高模型的表现能力和计算效率。同时提出了一种残差双注意模块（RDAB），将它应用在通道分割之后，保证雨纹被去除时背景图像不被破坏或模糊。由于雨纹之间存在较强的相关性，网络采取三阶段的学习策略，以便更好地捕捉雨纹在图像中的分布。TCSDFNet将去雨网络中的参数数量大大减少，更全面地提取了不同层次的雨纹特征，并在更大程度上减少背景图像细节的丢失。在合成数据集和真实数据集上的大量实验表明，所提出的网络实现了良好的去雨效果。

关键词: 雨纹, 通道分割, 图像去雨

Abstract: Single image deraining is a considerable challenge in the domain of image processing. The enhancement of image quality under adverse weather conditions heavily relies on the development of efficient single image rain streaks removal algorithms. However, prevailing rain streaks removal algorithms often suffer from an abundance of parameters. Furthermore, as the number of network layers increases, the extraction of image features tends to become redundant, detrimentally impacting both the efficacy of rain streaks removal and the preservation of background details. Therefore, a three-stage channel split dense fusion network (TCSDFNet) is proposed to obtain better rain removal performance. Specifically, multiple channel split blocks (CSB) are used in the network for dense feature fusion to gather the high and low level features of the rain streaks. CSB uses channel split operation to split the rainy image into multiple channels, and applies different rain streaks removal methods according to different levels of features to reduce redundant features and network parameters, and improve the performance ability and computational efficiency of the model. At the same time, the residual dual attention block (RDAB) is proposed, which is applied to the channel split to ensure that the background image will not be destroyed or blurred when the rain streaks are removed. In addition, due to the strong correlation between rain streaks, the network adopts a three-stage learning strategy in order to better capture the distribution of rain streaks in the image. TCSDFNet greatly reduces the number of parameters in the rain removal network, extracts the features of different levels more comprehensively, and reduces the loss of background image detail to a greater extent. A large number of experiments on synthetic datasets and real datasets show that the proposed network achieves good rain removal effect.

Key words: rain striations, channel segmentation, image deraining

张书婷, 王长月, 王长忠, 冷强奎. 单幅图像去雨的三阶段通道分割密集融合网络[J]. 计算机科学与探索, 2025, 19(3): 703-713.

ZHANG Shuting, WANG Changyue, WANG Changzhong, LENG Qiangkui. Three-Stage Channel Split Dense Fusion Network for Single Image Deraining[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(3): 703-713.

参考文献

[1] DENG S, WEI M Q, WANG J, et al. Detail-recovery image deraining via context aggregation networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 14548-14557.
[2] HE K M, GKIOXARI G, DOLLÁR P, et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2017: 2980-2988.
[3] SHAO Z F, WANG L G, WANG Z Y, et al. Saliency-aware convolution neural network for ship detection in surveillance video[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019, 30(3): 781-794.
[4] LIU H H, SHU N, TANG Q L, et al. Computational model based on neural network of visual cortex for human action recognition[J]. IEEE Transactions on Neural Networks and Learning Systems, 2018, 29(5): 1427-1440.
[5] PRASAD D K, RAJAN D, RACHMAWATI L, et al. Video processing from electro-optical sensors for object detection and tracking in a maritime environment: a survey[J]. IEEE Transactions on Intelligent Transportation Systems, 2017, 18(8): 1993-2016.
[6] HUANG D A, KANG L W, WANG Y F, et al. Self-learning based image decomposition with applications to single image denoising[J]. IEEE Transactions on Multimedia, 2013, 16(1): 83-93.
[7] KANG L W, LIN C W, FU Y H. Automatic single-image-based rain streaks removal via image decomposition[J]. IEEE Transactions on Image Processing, 2012, 21(4): 1742-1755.
[8] ZHANG H, PATEL V M. Convolutional sparse and low-rank coding-based rain streak removal[C]//Proceedings of the 2017 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2017: 1259-1267.
[9] LI Y, TAN R T, GUO X J, et al. Rain streak removal using layer priors[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2016: 2736-2744.
[10] FU X Y, HUANG J B, DING X H, et al. Clearing the skies: a deep network architecture for single-image rain removal[J]. IEEE Transactions on Image Processing, 2017, 26(6): 2944-2956.
[11] ZHENG Y P, YU X, ZHANG S L, et al. Residual multiscale based single image deraining[C]//Proceedings of the 30th British Machine Vision Conference 2019, 2019: 147.
[12] YASARLA R, PATEL V M. Uncertainty guided multi-scale residual learning-using a cycle spinning CNN for single image de-raining[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 8397-8406.
[13] QUE Y, LI S L, LEE H J. Attentive composite residual network for robust rain removal from single images[J]. IEEE Transactions on Multimedia, 2020, 23: 3059-3072.
[14] FAN Z W, WU H F, FU X Y, et al. Residual-guide network for single image deraining[C]//Proceedings of the 26th ACM International Conference on Multimedia. New York: ACM, 2018: 1751-1759.
[15] REN D W, ZUO W M, HU Q H, et al. Progressive image deraining networks: a better and simpler baseline[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 3932-3941.
[16] REN D W, SHANG W, ZHU P F, et al. Single image deraining using bilateral recurrent network[J]. IEEE Transactions on Image Processing, 2020, 29: 6852-6863.
[17] SHARMA S, TANG B, BALL J E, et al. Recursive multi-scale image deraining with sub-pixel convolution based feature fusion and context aggregation[J]. IEEE Access, 2020, 8: 177495-177505.
[18] LI M Y, WANG Y F, WANG C. Recursive residual atrous spatial pyramid pooling network for single image deraining[J]. Signal Processing: Image Communication, 2021, 99: 116430.
[19] PAN X M, YANG Y T, YANG C S, et al. Two-stage fusion model for heavy rain removal on single image[J]. Journal of Physics: Conference Series, 2021(1): 012041.
[20] CAO M, GAO Z, RAMESH B, et al. A two-stage density-aware single image deraining method[J]. IEEE Transactions on Image Processing, 2021, 30: 6843-6854.
[21] ZHANG H, PATEL V M. Density-aware single image de-raining using a multi-stream dense network[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 695-704.
[22] QIAN R, TAN R T, YANG W H, et al. Attentive generative adversarial network for raindrop removal from a single image[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2018: 2482-2491.
[23] WEI Y Y, ZHANG Z, WANG Y, et al. DerainCycleGAN: rain attentive CycleGAN for single image deraining and rainmaking[J]. IEEE Transactions on Image Processing, 2021, 30: 4788-4801.
[24] CAO M, GAO Z, RAMESH B, et al. Single image deraining integrating physics model and density-oriented conditional GAN refinement[J]. IEEE Signal Processing Letters, 2021, 28: 1635-1639.
[25] YE Y T, CHANG Y, ZHOU H Y, et al. Closing the loop: joint rain generation and removal via disentangled image translation[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 2053-2062.
[26] ZHAO J, XIE J Y, XIONG R Q, et al. Pyramid convolutional network for single image deraining[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 9-16.
[27] FU X Y, LIANG B R, HUANG Y, et al. Lightweight pyramid networks for image deraining[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(6): 1794-1807.
[28] JIANG K, WANG Z Y, YI P, et al. Multi-scale progressive fusion network for single image deraining[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 8343-8352.
[29] HUI Z, GAO X B, YANG Y C, et al. Lightweight image super-resolution with information multi-distillation network[C]//Proceedings of the 27th ACM International Conference on Multimedia. New York: ACM, 2019: 2024-2032.
[30] PRAJAPATI K, CHUDASAMA V, PATEL H, et al. Channel split convolutional neural network (ChaSNet) for thermal image super-resolution[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 4363-4372.
[31] HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: surpassing human-level performance on ImageNet classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Piscataway: IEEE, 2015: 1026-1034.
[32] YANG W H, TAN R T, FENG J S, et al. Deep joint rain detection and removal from a single image[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1685-1694.
[33] FU X Y, HUANG J B, ZENG D L, et al. Removing rain from single images via a deep detail network[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2017: 1715-1723.
[34] LI X, WU J L, LIN Z C, et al. Recurrent squeeze-and-excitation context aggregation net for single image deraining[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 262-277.
[35] YANG Y Z, LU H. Single image deraining using a recurrent multi-scale aggregation and enhancement network[C]//Proceedings of the 2019 IEEE International Conference on Multimedia and Expo. Piscataway: IEEE, 2019: 1378-1383.
[36] CHAI G Q, WANG Z B, GUO G D, et al. Recurrent attention dense network for single image de-raining[J]. IEEE Access, 2020, 8: 111278-111288.
[37] ZHANG Y, ZHANG J, HUANG B, et al. Single-image deraining via a recurrent memory unit network[J]. Knowledge-Based Systems, 2021, 218: 106832.
[38] TANG Q F, YANG J, LIU H B, et al. Single image deraining using context aggregation recurrent network[J]. Journal of Visual Communication and Image Representation, 2021, 75: 103039.
[39] ZHANG H, XIE Q Q, LU B, et al. Dual attention residual group networks for single image deraining[J]. Digital Signal Processing, 2021, 116: 103106.
[40] LI P P, JIN J Y, JIN G Y, et al. Deep scale-space mining network for single image deraining[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2022: 4275-4284.
[41] XIAO J, FU X Y, LIU A P, et al. Image de-raining transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2023, 45(11): 12978-12995.
[42] LI Y F, LU J Y, CHEN H M, et al. Dilated convolutional transformer for high-quality image deraining[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 4199-4207.
[43] ZHOU W N, YE L H, WANG X. Residual contextual hourglass network for single-image deraining[J]. Neural Processing Letters, 2024, 56(2): 63.
[44] ZAMIR S W, ARORA A, KHAN S, et al. Multi-stage progressive image restoration[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 14816-14826.
[45] WANG Z, WANF C, SU Z X, et al. Dense feature pyramid grids network for single image deraining[C]//Proceedings of the 2021 IEEE International Conference on Acoustics, Speech and Signal Processing. Piscataway: IEEE, 2021: 2025-2029.
[46] WEN Y B, GAO T, ZHANG J, et al. From heavy rain removal to detail restoration: a faster and better network[J]. Pattern Recognition, 2024, 148: 110205.

编辑推荐 0

Metrics

阅读次数

全文

HTML			PDF

最新录用	在线预览	正式出版	最新录用	在线预览	正式出版
0	0	0	20	0	35

来源	本网站	其他网站

次数	54	1
比例	98%	2%

摘要

最新录用	在线预览	正式出版

22	0	26

	来源	本网站

	次数	49
	比例	100%