No-Reference Low-Light Image Enhancement with Enhanced Feature Map

doi:10.3778/j.issn.1673-9418.2308052

Abstract

Abstract: Since low-light images are of poor quality and carry noise, resulting in insufficient contrast and brightness, unclear details in the image, and the problem of high cost of obtaining paired low-light image datasets, a no-reference low-light image enhancement method for feature map enhancement is proposed under the influence of the Mach band effect of biological vision. Firstly, the model uses the enhanced filter block (EFB) to enhance the feature of images and feature maps, which can suppress noise and enhance the feature details to improve the ability of the network to learn features. Then, skip connection is combined with efficient spatial attention (ESA) module to extract global context information and local regional features by fusing enhanced shallow features and deep features, which effectively retains image color information, avoids detail loss, and improves the generalization ability of the network. Finally, the pixel estimation curve is used to adjust the dynamic range of the pixels in the low-light image and enhance the brightness of the low-light image. Experimental results show that the PSNR, SSIM, LPIPS and NIQE of the image processed by the algorithm reach 17.709 dB, 0.657, 0.239 and 3.486 respectively. Compared with the existing mainstream algorithms, it can better achieve the purpose of image enhancement, and effectively improves the image brightness and detail information, while maintaining the naturalness of the image.

Key words: image enhancement, biological vision mechanism, enhanced feature map, no-reference method, attention mechanism, image processing

摘要： 针对低光照图像质量不佳、夹杂噪声导致对比度和亮度不足、细节不清晰，且成对的低光照图像数据集获取成本过高的问题，在生物视觉马赫带效应的启发下，提出一种强化特征图的无参考低光照图像增强方法。使用强化滤波块（EFB）对图像和特征图进行特征强化，抑制噪声的同时强化特征细节，提高网络对特征的学习能力。将跳跃连接与空间注意力模块（ESA）结合，通过融合强化的浅层特征与深层特征来提取全局上下文信息和局部区域特征，有效保留了图像的色彩信息，避免细节丢失，提高网络的泛化能力。使用像素估计曲线调整低光照图像像素的动态范围，对其进行亮度增强。实验结果表明，经该算法处理后的图像在PSNR、SSIM、LPIPS和NIQE等指标上分别达到了17.709 dB、0.657、0.239和3.486，该方法相较于现有的主流算法能够更好地达到图像增强目的，有效地提升图像亮度和细节信息，同时保持图像的自然属性。

关键词: 图像增强, 生物视觉机制, 强化特征图, 无参考方法, 注意力机制, 图像处理

YUAN Heng, WANG Xiaoxue, ZHANG Shengchong. No-Reference Low-Light Image Enhancement with Enhanced Feature Map[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(9): 2449-2465.

袁姮, 王笑雪, 张晟翀. 强化特征图的无参考低光照图像增强[J]. 计算机科学与探索, 2024, 18(9): 2449-2465.

References

[1] 赵文清, 康怿瑾, 赵振兵, 等. 改进YOLOv5s的遥感图像目标检测[J]. 智能系统学报, 2023, 18(1): 86-95.
ZHAO W Q, KANG Y J, ZHAO Z B, et al. A remote sensing image object detection algorithm with improved YOLOv5s[J]. CAAI Transactions on Intelligent Systems, 2023, 18(1): 86-95.
[2] PISANO E D, ZONG S, HEMMINGER B M, et al. Contrast limited adaptive histogram equalization image processing to improve the detection of simulated specultions in dense mammograms[J]. Journal of Digital Imaging, 1998, 11(4): 193-200.
[3] IRAHIM H, KONG N S P. Brightness preserving dynamic histogram equalization for image contrast enhancment[J]. IEEE Transactions on Consumer Electronics, 2007, 53(4): 1752-1758.
[4] 江巨浪, 刘国明, 朱柱, 等. 基于快速模糊聚类的动态多直方图均衡化算法[J]. 电子学报, 2022, 50(1): 167-176.
JIANG J L, LIU G M, ZHU Z, et al. Dynamic multi-histogram equalization algorithm based on fast fuzzy clustering[J]. Acta Electronica Sinica, 2022, 50(1): 167-176.
[5] JEBADASS J R, BALASUBRAMANIAM P. Low contrast enhancement technique for color images using interval valued intuitionistic fuzzy sets with contrast limited adaptive histogram equalization[J]. Soft Computing: A Fusion of Foun-dations, Methodologies and Applications, 2022, 26(10): 4949-4960.
[6] ZUIDERVELD K. Contrast limited adaptive histogram equa-lization[J]. Graphics Gems, 1994: 474-485.
[7] 侯利霞, 聂丰英, 万里勇. 多尺度自适应Gamma矫正的低照图像增强[J]. 云南大学学报(自然科学版), 2023, 45(1): 57-66.
HOU L X, NIE F Y, WAN L Y. Multiscale adaptive Gamma correction for low-light image enhancement[J]. Journal of Yunnan University (Natural Sciences Edition), 2023, 45(1): 57-66.
[8] DONG X, PANG Y A, WEN J G. Fast efficient algorithm for enhancement of low lighting video[C]//Proceedings of the 2011 IEEE International Conference on Multimedia & Expo, Barcelona, Jul 11-15, 2011. Piscataway: IEEE, 2011: 1-6.
[9] 王硕, 陈金玉. 自适应校正透射率的暗通道先验去雾算法[J]. 计算机工程与应用, 2021, 57(13): 207-211.
WANG S, CHEN J Y. Dark channel prior defogging algorithm for adaptive correction transmittance[J]. Computer Engineering and Applications, 2021, 57(13): 207-211.
[10] 王欣, 徐平平, 吴菲. 基于指数同态滤波耦合细节锐化规则的红外图像增强算法[J]. 电子测量与仪器学报, 2021, 35(10): 9-16.
WANG X, XU P P, WU F. Infrared image enhancement algorithm based on exponential homomorphic filtering coupled with detail sharpening rule[J]. Journal of Electronic Measurement and Instrumentation, 2021, 35(10): 9-16.
[11] 谢晓方, 刘厚君, 张龙杰, 等. 基于渐进式生成对抗网络的舰船红外图像仿真[J]. 激光与红外, 2021, 51(4): 471-479.
XIE X F, LIU H J, ZHANG L J, et al. Infrared image simulation for warship based on progressive generative adversarial nework[J]. Laser & Infrared, 2021, 51(4): 471-479.
[12] 田文利. 基于双重滤波与锐化的遥感图像增强算法[J]. 国外电子测量技术, 2017, 36(4): 13-16.
TIAN W L. Remote sensing image enhancement algorithm based on double filtering and sharpening[J]. Foreign Electronic Measurement Technology, 2017, 36(4): 13-16.
[13] 钱军, 万里勇. 引导滤波与像素重分布的低照图像增强[J]. 光电子·激光, 2023, 34(11): 1169-1177.
QIAN J, WAN L Y. Low-light image enhancement based on guided filtering and pixel redistribution[J]. Journal of Optoelectronics·Laser, 2023, 34(11): 1169-1177.
[14] 李胜勋, 黄靖. 基于Gabor滤波和Scharr算法的手掌静脉图像增强方法[J]. 北京生物医学工程, 2023, 42(5): 488-495.
LI S X，HUANG J. Palm vein image enhancement method based on Gabor filter and Scharr algorithm[J]. Beijing Biomedical Engineering, 2023, 42(5): 488-495．
[15] GUO X J, LI Y, LING H B. LIME: low-light image enhancement via illumination map estimation[J]. IEEE Transactions on Image Processing, 2017, 26(2): 982-993.
[16] LAND E H, MCCANN J J. Lightness and retinex theory[J]. Journal of the Optical Society of America, 1971, 61(1): 1-11.
[17] JOBSON D J, RAHMAN Z, WOODELL G A. Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing, 1997, 6(3): 451-462.
[18] RAHMAN Z, JOBSON D J, WOODELL G A. Multiscale retinex for color image enhancement[C]//Proceedings of the 3rd IEEE International Conference on Image Processing, Lausanne, Sep 16-19, 1996. Piscataway: IEEE, 1996: 1003-1006.
[19] RAHMAN Z, JOBSON D J, WOODELL G A. Retinex processing for automatic image enhancement[J]. Journal of Electronic Imaging, 2004, 13(1): 100-110.
[20] LI M D, LIU J Y, YANG W H, et al. Structure-revealing low-light image enhancement via robust retinex model[J]. IEEE Transactions on Image Processing, 2018, 27(6): 2828-2841.
[21] LORE K G, AKINTAYO A, SARKAR S. LLNet: a deep auto-encoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650-662.
[22] WEI C, WANG W, YANG W, et al. Deep retinex decomposition for low-light enhancement[EB/OL]. [2023-06-13]. https:// arxiv.org/abs/1808.04560.
[23] WU W, WENG J, ZHANG P, et al. URetinex-Net: retinex-based deep unfolding network for low-light image enhancement[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, Jun 19-20, 2022. Piscataway: IEEE, 2022: 5901-5910.
[24] LV F, LU F, WU J, et al. MBLLEN: low-light image/video enhancement using CNNs[C]//Proceedings of the 2018 British Machine Vision Conference, Newcastle, Sep 2-6, 2018.Cham: Springer, 2018: 4.
[25] WANG Y, WAN R, YANG W, et al. Low-light image enhancement with normalizing flow[EB/OL]. [2023-06-13]. https://arxiv.org/abs/2109.05923.
[26] COTOGNI M, CUSANO C. TreEnhance: a tree search met-hod for low-light image enhancement[J]. Pattern Recognition, 2023, 136: 109249.
[27] ZHANG Y, LIU H, DING D. A cross-scale framework for low-light image enhancement using spatial-spectral information[J]. Computers and Electrical Engineering, 2023, 106: 108608-108621.
[28] 林荐壮, 杨文忠, 谭思翔, 等. 融合滤波增强和反转注意力网络用于息肉分割[J]. 计算机应用, 2023, 43(1): 265-272.
LIN J Z, YANG W Z, TAN S X, et al. Fusing filter enhancement and reverse attention network for polyp segmentation[J]. Journal of Computer Applications, 2023, 43(1): 265-272.
[29] 徐文晨, 樊佳庆, 宋慧慧. 基于上下文Transformer的低光照图像增强网络[J]. 计算机与数字工程, 2023, 51(1): 237-244.
XU W C, FAN J Q, SONG H H. Low-light image enhancement network based on context Transformer[J]. Computer and Digital Engineering, 2023, 51(1): 237-244.
[30] 薛金强, 吴秦. 面向图像复原和增强的轻量级交叉门控Transformer[J]. 计算机科学与探索, 2024, 18(3): 718-730.
XUE J Q, WU Q. Lightweight cross-gating Transformer for image restoration and enhancement[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(3): 718-730.
[31] JIANG Y, GONG X, LIU D, et al. EnlightenGAN: deep light enhancement without paired supervision[J]. IEEE Trans-actions on Image Processing, 2021, 30: 2340-2349.
[32] ZHU A, ZHANG L, SHEN Y, et al. Zero-shot restoration of underexposed images via robust retinex decomposition[C]//Proceedings of the 2020 IEEE International Conference on Multimedia and Expo, London, Jul 6-10, 2020. Piscataway: IEEE, 2020: 1-6.
[33] ZHANG F, SHAO Y, SUN Y, et al. Unsupervised low-light image enhancement via histogram equalization prior[EB/OL]. [2023-06-13]. https://arxiv.org/abs/2112.01766.
[34] MA L, MA T, LIU R, et al. Toward fast, flexible, and robust low-light image enhancement[C]//Proceedings of the 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, New Orleans, Jun 19-24, 2022. Piscataway: IEEE, 2022: 5637-5646.
[35] GUO C, LI C, GUO J, et al. Zero-reference deep curve estimation for low-light image enhancement[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 1780-1789.
[36] NIELSEN C J. Effect of scenario and experience on interpretation of Mach bands[J]. Journal of Endodontics, 2001, 27(11): 687-691.
[37] 傅玉川, 王植恒, 万海峰. 用广义Gabor函数的感受野模型描述的马赫带[J]. 生物化学与生物物理进展, 1993, 20(6): 478-479.
FU Y C, WANG Z H, WAN H F. Machband described by the receptive field model of generalized Gabor function[J]. Progress in Biochemistry and Biophysics, 1993, 20(6): 478-479.
[38] GUNTER K, THOMAS U, ANDREAS M, et al. Self-normalizing neural networks[EB/OL]. [2023-06-13]. https://arxiv.org/abs/1706.02515.
[39] YUAN L, SUN J. Automatic exposure correction of consumer photographs[C]//Proceedings of the 12th European Conference on Computer Vision, Florence, Oct 7-13, 2012. Berlin, Heidelberg: Springer, 2012: 771-785.
[40] ZHANG L, ZHANG L, LIU X, et al. Zero-shot restoration of backlit images using deep internal learning[C]//Proceedings of the 27th ACM International Conference on Multi-media, Nice, Oct 21-25, 2019. New York: ACM, 2019: 1623-1631.
[41] CAI J, GU S, ZHANG L. Learning a deep single image contrast enhancer from multi-exposure images[J]. IEEE Transactions on Image Processing, 2018, 27(4): 2049-2062.
[42] ZHANG R, ISOLA P, EFROS A A, et al. The unreasonable effectiveness of deep features as a perceptual metric[EB/OL]. [2023-06-13]. https://arxiv.org/abs/1801.03924.