LEGAN: New Dark and Weak Light Image Enhancement Algorithm

doi:10.3778/j.issn.1673-9418.2310007

Abstract

Abstract: In order to solve the problems of low brightness, contrast, signal-to-noise ratio and high noise pollution in dark and weak light images, this paper proposes a new dark and weak light image enhancement algorithm LEGAN (low-light enhancement generative adversarial network). This algorithm first inputs the image into the proposed Gamma curve estimation network to obtain the feature map containing Gamma parameters, then enhances the brightness through LEB (light enhancement block) module, and iteratively enhances the result by cascading LEB. Then, a global-local discriminator structure based on PatchGAN is used to improve image resolution and recover details. Finally, the gap between the true label and the output result is limited by introducing a perceived loss. The lighting smoothness loss is used to maintain the monotonic relationship between adjacent pixels. Meanwhile, the spatial consistency loss is combined to enhance the spatial correlation of images. Experimental results show that compared with most of the current mainstream enhancement algorithms, the proposed algorithm has a higher degree of detail restoration, and can effectively avoid the problem of ill illumination in the local region of enhanced image.

Key words: dark and weak light, image enhancement, Gamma curve estimation network, global-local discriminator, loss function

摘要： 针对暗弱光照图像所存在的亮度、对比度、信噪比低，以及噪声污染大等问题，提出了一种新的暗弱光照图像增强算法LEGAN。该算法将图像输入至所提伽马曲线估计网络求得包含伽马参数的特征图，再经过LEB模块增强亮度，并通过级联LEB的方式迭代增强结果。采用基于PatchGAN的全局-局部判别器结构来提高图像分辨率和恢复图像细节。通过引入感知损失来限制真实标签和输出结果之间的差距，利用照明平滑度损失保持相邻像素之间的单调性关系，同时结合空间一致性损失来增强图像的空间相关性。实验结果表明，相比于现今大多数主流增强算法，该算法的细节还原度相对较高，且能有效避免增强后的图像出现局部亮度不佳等问题。

关键词: 暗弱光照, 图像增强, 伽马曲线估计网络, 全局-局部判别器, 损失函数

GUO Fan, LIU Wentao, LI Xiaohu, TANG Jin. LEGAN: New Dark and Weak Light Image Enhancement Algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(9): 2422-2435.

郭璠, 刘文韬, 李小虎, 唐琎. LEGAN：一种新的暗弱光照图像增强算法[J]. 计算机科学与探索, 2024, 18(9): 2422-2435.

References

[1] 江泽涛, 钱艺, 伍旭, 等. 一种基于ARD-GAN的低照度图像增强方法[J]. 电子学报, 2021, 49(11): 2160-2165.
JIANG Z T, QIAN Y, WU X, et al. Low-light image enhancement method based on ARD-GAN[J]. Acta Electronica Sinica, 2021, 49(11): 2160-2165.
[2] LAND E H. The retinex[J]. American Scientist, 1964, 52(2): 247-264.
[3] JOBSON D J, RAHMAN Z, WOODELL G A. Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing, 1997, 6(3): 451-462.
[4] 常戬, 韩旭. 结合导向滤波与自适应算子的水下图像增强[J]. 计算机工程与应用, 2023, 59(4): 216-223.
CHANG J, HAN X. Underwater image enhancement combining guide filtering with adaptive operator[J]. Computer Engineering and Applications, 2023, 59(4): 216-223.
[5] LORE K G, AKINTAYO A, SARKAR S. LLNet: a deep autoencoder approach to natural low-light image enhancement[J]. Pattern Recognition, 2017, 61: 650-662.
[6] WEI C, WANG W, YANG W, et al. Deep retinex decomposition for low-light enhancement[C]//Proceedings of the British Machine Vision Conference 2018, Newcastle, Sep 3-6, 2018: 155.
[7] WANG R, ZHANG Q, FU C W, et al. Underexposed photo enhancement using deep illumination estimation[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 6849-6857.
[8] JIANG Y, GONG X, LIU D, et al. EnlightenGAN: deep light enhancement without paired supervision[J]. IEEE Transactions on Image Processing, 2021, 30: 2340-2349.
[9] 徐少平, 陈孝国, 李芬, 等. 采用两阶段混合策略实现的低照度图像增强算法[J]. 电子学报, 2021, 49(11): 2166-2170.
XU S P, CHEN X G, LI F, et al. A low-light image enhancement algorithm using two-stage hybrid[J]. Acta Electronica Sinica, 2021, 49(11): 2166-2170.
[10] 卫依雪, 周冬明, 王长城, 等. 结合多分支结构和U-net的低照度图像增强[J]. 计算机工程与应用, 2022, 58(12): 199-208.
WEI Y X, ZHOU D M, WANG C C, et al. Low-light image enhancement using multi-branch structure and U-net[J]. Computer Engineering and Applications, 2022, 58(12): 199-208.
[11] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144.
[12] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2017: 2223-2232.
[13] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 3-19.
[14] JOLICOEUR-MARTINEAU A. The relativistic discriminator: a key element missing from standard GAN[C]//Proceeding of the 7th International Conference on Learning Representations,New Orleans, May 6-9, 2019.
[15] MAO X, LI Q, XIE H, et al. Least squares generative adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2017: 2794-2802.
[16] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[C]//roceedings of the 3rd International Conference on Learning Representations, San Diego, May 7-9, 2015.
[17] BYCHKOVSKY V, PARIS S, CHAN E, et al. Learning photographic global tonal adjustment with a database of input/output image pairs[C]//Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2011: 97-104.
[18] 黄颖, 彭慧, 李昌盛, 等. LLFlowGAN: 以生成对抗方式约束可逆流的低照度图像增强[J]. 中国图象图形学报, 2024, 29(1): 65-79.
HUANG Y, PENG H, LI C S, et al. LLFlowGAN： a low-light image enhancement method for constraining invertible flow in a generative adversarial manner[J]. Journal of Image and Graphics, 2024, 29(1): 65-79.
[19] 杨镇雄, 谭台哲. 基于生成对抗网络的低光照图像增强算法[J]. 广东工业大学学报, 2024, 41(1): 55-62.
YANG Z X, TAN T Z. Low illumination image enhancement algorithm based on generative adversarial network[J]. Journal of Guangdong University of Technology, 2024, 41(1): 55-62.
[20] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[21] MITTAL A, SOUNDARARAJAN R, BOVIK A C. Making a “completely blind” image quality analyzer[J]. IEEE Signal Processing Letters, 2013, 20(1): 209-212.
[22] ISOLA P, ZHU J Y, ZHOU T, et al. Image-to-image translation with conditional adversarial networks[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition.Washington: IEEE Computer Society, 2017: 1125-1134.
[23] ZHAO Z, XIONG B, WANG L, et al. RetinexDIP: a unified deep framework for low-light image enhancement[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2021, 32(3): 1076-1088.