Dual-channel Quaternion Convolutional Network for Denoising

doi:10.3778/j.issn.1673-9418.2109042

Abstract

Abstract: The color image denoising based on deep learning usually uses convolution on each channel, and then merges multi-channel data into single channel data. This method does not fully consider the spectral correlation between color channels, which may casuse distortion of the denoising results. Quaternion convolution can solve this problem by treating a color pixel as a whole. However, a single quaternion convolutional network can not restore the image details well. To solve the problem, a dual-channel quaternion convolutional network (DQNet) for color random impulse noise removal is proposed. Firstly, according to the strategy of structure channel and color channel fusion, a structure detail restoration network based on dilated convolution is proposed to obtain structure and edge features, and quaternion convolution network is used to extract cross-channel color information. Secondly, aiming at the problem that convolution operation will cause partial global information loss, the long line connection is used to fuse the input noise image with the convolution results, and then, a feature enhancement module based on attention mechanism is designed to guide the network to extract potential noise features from complex background. Finally, the residual learning is used to achieve the restoration of color random impulse noise. Experimental results show that the proposed algorithm has better denoising performance, especially in moderate noise level or high noise level.

Key words: image denoising, deep learning, random impulsive noise, quaternion convolution, dual-channel

摘要： 基于深度学习的彩色图像去噪方法通常是在各个通道进行卷积操作后再进行合并得到最后的卷积结果。这种方式没有充分考虑色彩通道之间的光谱相关性，可能导致去噪结果的失真。四元数卷积将彩色像素当作一个整体来进行处理，可以很好地解决这一问题。但是单一的四元数卷积网络不能较好地还原图像细节信息。针对这一问题，提出一种用于去除彩色随机脉冲噪声的双通道四元数卷积网络（DQNet）。该网络首先基于结构通道和色彩通道融合的策略，采用基于扩张卷积的结构细节还原模块提取结构和边缘特征，采用四元数卷积网络提取色彩特征；然后针对卷积运算会导致部分全局信息丢失的问题，通过长线连接将含有丰富全局特征的输入噪声图像与卷积结果进行融合，设计基于注意力机制的特征增强模块来指导网络提取复杂背景中的潜在噪声特征；最后利用残差学习实现彩色随机脉冲噪声的复原。实验结果表明，所提算法具有较好的去噪性能，在中度噪声污染或高度噪声污染的情况下去噪效果更为突出。

关键词: 图像去噪, 深度学习, 随机脉冲噪声, 四元数卷积, 双通道

CAO Yiqin, RAO Zhechu, ZHU Zhiliang, WAN Sui. Dual-channel Quaternion Convolutional Network for Denoising[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1359-1372.

曹义亲, 饶哲初, 朱志亮, 万穗. 双通道四元数卷积网络去噪方法[J]. 计算机科学与探索, 2023, 17(6): 1359-1372.

References

[1] ILESANMI A E, ILESANMI T O. Methods for image de-noising using convolutional neural network: a review[J]. Complex & Intelligent Systems, 2021, 7: 2179-2198.
[2] NADEEM M, HUSSAIN A, MUNIR A, et al. Removal of random valued impulse noise from grayscale images using quadrant based spatially adaptive fuzzy filter[J]. Signal Processing, 2020, 169: 107403.
[3] DONG Y, CHAN R H, XU S. A detection statistic for random-valued impulse noise[J]. IEEE Transactions on Image Pro-cessing, 2007, 16(4): 1112-1120.
[4] LYU Q, GUO M, PEI Z. DeGAN: mixed noise removal via generative adversarial networks[J]. Applied Soft Compu-ting, 2020, 95: 106478.
[5] CHAN R H, HU C, NIKOLOVA M. An iterative procedure for removing random-valued impulse noise[J]. IEEE Signal Processing Letters, 2004, 11(12): 921-924.
[6] KIM S. PDE-based image restoration: a hybrid model and color image denoising[J]. IEEE Transactions on Image Pro-cessing, 2006, 15(5): 1163-1170.
[7] BROOK A. Three-dimensional wavelets-based denoising of hyperspectral imagery[J]. Journal of Electronic Imaging, 2015, 24(1): 013034.
[8] ASTOLA J, HAAVISTO P, NEUVO Y. Vector median filters[J]. Proceedings of the IEEE, 1990, 78(4): 678-689.
[9] TRAHANIAS P E, VENETSANOPOULOS A N. Vector direc-tional filters—a new class of multichannel image proces-sing filters[J]. IEEE Transactions on Image Processing, 1993, 2(4): 528-534.
[10] KARAKOS D G, TRAHANIAS P E. Generalized multichannel image-filtering structures[J]. IEEE Transactions on Image Processing, 1997, 6(7): 1038-1045.
[11] SMOLKA B, CHYDZINSKI A. Fast detection and impul-sive noise removal in color images[J]. Real-Time Imaging, 2005, 11(5/6): 389-402.
[12] CELEBI M E, ASLANDOGAN Y A. Robust switching vec-tor median filter for impulsive noise removal[J]. Journal of Electronic Imaging, 2008, 17(4): 043006.
[13] XU J, WANG L, SHI Z. A switching weighted vector me-dian filter based on edge detection[J]. Signal Processing, 2014, 98: 359-369.
[14] ROY A, LASKAR R H. Impulse noise removal based on SVM classification[C]//Proceedings of the IEEE Region 10 International Conference TENCON, Macao, China, Nov 1-4, 2015. Piscataway: IEEE, 2015: 1-5.
[15] SA P K, MAJHI B. An improved adaptive impulsive noise sup-pression scheme for digital images[J]. AEU-International Journal of Electronics and Communications, 2010, 64(4): 322-328.
[16] TURKMEN I. The ANN based detector to remove ran-domvalued impulse noise in images[J]. Journal of Visual Communication and Image Representation, 2016, 34: 28-36.
[17] ZHANG K, ZUO W, CHEN Y, et al. Beyond a Gaussian de-noiser: residual learning of deep CNN for image denoising[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3142-3155.
[18] ZHANG K, ZUO W, ZHANG L. FFDNet: toward a fast and flexible solution for CNN-based image denoising[J]. IEEE Tran-sactions on Image Processing, 2018, 27(9): 4608-4622.
[19] LEFKIMMIATIS S. Universal denoising networks: a novel CNN architecture for image denoising[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Computer Society, 2018: 3204-3213.
[20] CHEN J, CHEN J, CHAO H, et al. Image blind denoising with generative adversarial network based noise modeling[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Computer Society, 2018: 3155-3164.
[21] RADLAK K, MALINSKI L, SMOLKA B. Deep learning for impulsive noise removal in color digital images[C]//Pro-ceedings of the Real-Time Image Processing and Deep Lear-ning 2019, Baltimore, May 14, 2019. Bellingham: SPIE, 2019: 1099608.
[22] RADLAK K, MALINSKI L, SMOLKA B. Deep learning based switching filter for impulsive noise removal in color images[J]. Sensors, 2020, 20(10): 2782.
[23] ZHANG W, JIN L, SONG E, et al. Removal of impulse noise in color images based on convolutional neural network[J]. Applied Soft Computing, 2019, 82: 105558.
[24] ZHANG Y, TIAN Y, KONG Y, et al. Residual dense net-work for image super-resolution[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recogni-tion, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Com-puter Society, 2018: 2472-2481.
[25] TIAN C, XU Y, LI Z, et al. Attention-guided CNN for image denoising[J]. Neural Networks, 2020, 124: 117-129.
[26] GENG X, HU X, XIAO J. Quaternion switching filter for impulse noise reduction in color image[J]. Signal Proces-sing, 2012, 92(1): 150-162.
[27] WANG G, LIU Y, ZHAO T. A quaternion-based switching filter for colour image denoising[J]. Signal Processing, 2014, 102: 216-225.
[28] JIN L, ZHU Z, SONG E, et al. An effective vector filter for impulse noise reduction based on adaptive quaternion color distance mechanism[J]. Signal Processing, 2019, 155: 334-345.
[29] HAMILTON W R. Elements of quaternions[M]. New York: Cambridge University Press, 1866.
[30] ZHU X Y, XU Y, XU H T, et al. Quaternion convolutional neural networks[C]//LNCS 11212: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 645-661.
[31] LIU R, SHEN J, WANG H, et al. Enhanced 3D human pose estimation from videos by using attention-based neural net-work with dilated convolutions[J]. International Journal of Computer Vision, 2021, 129(5): 1596-1615.
[32] ARBELAEZ P, MAIRE M, FOWLKES C, et al. Contour detection and hierarchical image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2010, 33(5): 898-916.
[33] FRANZEN R. Kodak lossless true color image suite[EB/OL]. [2021-07-21]. http://r0k.us/graphics/kodak.
[34] LI X, GUNTURK B, ZHANG L. Image demosaicing: a systematic survey[C]//Proceedings of the Visual Commu-nications and Image Processing 2008, San Jose, Jan 28, 2008. San Francisco: SPIE, 2008: 68221J.
[35] XU J, LI H, LIANG Z, et al. Real-world noisy image de-noising: a new benchmark[J]. arXiv:1804.02603, 2018.
[36] MA Z, WU H R, QIU B. A robust structure-adaptive hybrid vector filter for color image restoration[J]. IEEE Transac-tions on Image Processing, 2005, 14(12): 1990-2001.
[37] LUKAC R, PLATANIOTIS K N. A taxonomy of color image filtering and enhancement solutions[J]. Advances in Im-aging and Electron Physics, 2006, 140: 188.
[38] ZHANG L, ZHANG L, MOU X, et al. FSIM: a feature simi-larity index for image quality assessment[J]. IEEE Transac-tions on Image Processing, 2011, 20(8): 2378-2386.
[39] ZHONG L, IBRAHIM M T, ZHANG Y, et al. Hybrid vector filters based on marginal ordering for impulsive noise suppres-sion in color images[J]. Journal of Signal Processing Systems, 2017, 89(2): 379-394.
[40] YUAN G, GHANEM B. L0TV: a new method for image res-toration in the presence of impulse noise[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington:IEEE Computer Society, 2015: 5369-5377.
[41] JIN L, LIU H, XU X, et al. Color impulsive noise removal based on quaternion representation and directional vector order-statistics[J]. Signal Processing, 2011, 91(5): 1249-1261.
[42] JIN L, ZHU Z, XU X, et al. Two-stage quaternion switching vector filter for color impulse noise removal[J]. Signal Pro-cessing, 2016, 128: 171-185.
[43] ZHU Z, JIN L, SONG E, et al. Quaternion switching vector median filter based on local reachability density[J]. IEEE Signal Processing Letters, 2018, 25(6): 843-847.