Super-Resolution Reconstruction Algorithm of Remote Sensing Image with Two-Branch Semantic Enhanced Perception

doi:10.3778/j.issn.1673-9418.2303044

Abstract

Abstract: Aiming at the problem of poor reconstruction due to the blurring of feature targets in remote sensing images and the influence of background noise, in this paper, a super-resolution reconstruction algorithm for remote sensing images incorporating two-branch semantic enhanced perception is proposed. Firstly, a global-local spatial attention module is designed to enhance the semantic representation of features at different scales of spatial global-local, and at the same time strengthen the discriminative ability of the network for effective feature groups. Secondly, a channel grouping-aggregation attention module is proposed to enhance the model’s discriminative ability for ground objects features by designing feature grouping-aggregation and channel attention modules, and strengthen the model’s ability to focus on effective feature channels. Experiments show that on the UC Merced dataset, the PSNR reaches 34.397 dB, 29.920 dB and 28.128 dB respectively at the ×2/×3/×4 multiplier, and the structural similarity reaches 0.931, 0.834 and 0.791 at the ×2/×3/×4 multiplier. On the AID dataset, the PSNR reaches 32.524 dB, 29.317 dB and 27.522 dB respectively at the×2/×3/×4 multiplier, and the structural similarity reaches 0.895, 0.829 and 0.721 at the ×2/×3/×4 multiplier. Compared with other mainstream algorithms, both indices are improved, and the edge and regional details of reconstructed images are better, effectively overcoming the problems of fuzzy feature information of ground objects and background noise, which lead to poor reconstruction effect of remote sensing images.

Key words: remote sensing image, spatial attention, channel attention, super-resolution reconstruction

摘要： 针对遥感图像中地物目标的特征信息模糊以及背景噪声影响导致遥感图像重建效果差的问题，提出一种融合双分支语义增强感知的遥感图像超分辨率重建算法。首先，设计了一种全局-局部空间注意力模块，该模块用于增强特征在空间全局-局部不同尺度下的语义表征能力，同时强化网络对有效特征组的分辨能力；其次，提出一种通道分组-聚合注意力模块，通过设计特征分组-聚合以及通道注意力模块，增强模型对地物目标特征的区分，强化对有效特征通道的关注能力。实验表明，所提算法在UC Merced数据集上，峰值信噪比在×2/×3/×4倍率下分别达到了34.397 dB、29.920 dB和28.128 dB，结构相似度在×2/×3/×4倍率下达到了0.931、0.834和0.791。在AID数据集上，峰值信噪比在×2/×3/×4倍率下分别达到了32.524 dB、29.317 dB和27.522 dB，结构相似度在×2/×3/×4倍率下达到了0.895、0.829和0.721。两个指标相较于等主流算法均有所提升，重建后图像的边缘与区域细节效果更优，有效克服了地物目标的特征信息模糊及背景噪声影响导致遥感图像重建效果差的问题。

关键词: 遥感图像, 空间注意力, 通道注意力, 超分辨率重建

WANG Chaoxue, DAI Ning. Super-Resolution Reconstruction Algorithm of Remote Sensing Image with Two-Branch Semantic Enhanced Perception[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(5): 1271-1285.

王超学, 代宁. 融合双分支语义增强感知的遥感图像超分辨率重建算法[J]. 计算机科学与探索, 2024, 18(5): 1271-1285.

References

[1] 沈焕锋, 艾廷华, 刘耀林, 等. 影像超分辨率重建技术的发展与应用现状[J]. 测控技术, 2009, 28(6): 7.
SHEN H F, AI T H, LIU Y L, et al. Development and application of super resolution image reconstruction technique[J]. Measurement & Control Technology, 2009, 28(6): 7.
[2] GLASNER D, BAGON S, IRANI M. Super-resolution from a single image[C]//Proceedings of the 2009 IEEE 12th International Conference on Computer Vision. Piscataway: IEEE, 2009: 349-356.
[3] RHEE S, KANG M G. Discrete cosine transform based regularized high-resolution image reconstruction algorithm[J]. Optical Engineering, 1999, 38(8): 1348-1356.
[4] ELAD M, FEUER A. Restoration of a single superresolution image from several blurred, noisy, and undersampled measured images[J]. IEEE Transactions on Image Processing, 1997, 6(12): 1646-1658.
[5] STARK H, OSKOUI P. High-resolution image recovery from image-plane arrays, using convex projections[J]. Journal of the Optical Society of America A, 1989, 6(11): 1715-1726.
[6] DONG C, LOY C C, TANG X. Accelerating the super-resolution convolutional neural network[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 391-407.
[7] LIEBEL L, K?RNER M. Single-image super resolution for multispectral remote sensing data using convolutional neural networks[J]. International Archives of the Photogrammetry, Remote Sensing & Spatial Information Sciences, 2016, 41.
[8] LEI S, SHI Z, ZOU Z. Super-resolution for remote sensing images via local-global combined network[J]. IEEE Geo-science and Remote Sensing Letters, 2017, 14(8): 1243-1247.
[9] LIM B, SON S, KIM H, et al. Enhanced deep residual networks for single image super-resolution[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2017: 136-144.
[10] KIM J, LEE J K, LEE K M. Accurate image super-resolution using very deep convolutional networks[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 1646-1654.
[11] HAUT J M, PAOLETTI M E, FERNáNDEZ-BELTRAN R, et al. Remote sensing single-image superresolution based on a deep compendium model[J]. IEEE Geoscience and Remote Sensing Letters, 2019, 16(9): 1432-1436.
[12] XU W, XU G L, WANG Y, et al. High quality remote sensing image super-resolution using deep memory connected network[C]//Proceedings of the 2018 IEEE International Geoscience and Remote Sensing Symposium. Piscataway: IEEE, 2018: 8889-8892.
[13] 张帅勇, 刘美琴, 姚超, 等. 分级特征反馈融合的深度图像超分辨率重建[J]. 自动化学报, 2022, 48(4): 992-1003.
ZHANG S Y, LIU M Q, YAO C, et al. Hierarchical feature feedback network for depth super-resolution reconstruction[J]. Acta Automatica Sinica, 2022, 48(4): 992-1003.
[14] 章伟帆, 曾庆鹏. 多重放大的医学图像超分辨率重建[J]. 计算机工程与应用, 2022, 58(23): 230-237.
ZHANG W F, CENG Q P. Multi-scale medical image super-resolution reconstruction[J]. Computer Engineering and Applications, 2022, 58(23): 230-237.
[15] HU J, SHEN L, SUN G. Squeeze-and-excitation networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 7132-7141.
[16] WANG X, GIRSHICK R, GUPTA A, et al. Non-local neural networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2018: 7794-7803.
[17] ZHANG Y, LI K, LI K, et al. Image super-resolution using very deep residual channel attention networks[C]//Proceedings of the 15th European Conference on Computer Vision. Cham: Springer, 2018: 286-301.
[18] DAI T, CAI J, ZHANG Y, et al. Second-order attention network for single image super-resolution[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2019: 11065-11074.
[19] MEI Y, FAN Y, ZHOU Y, et al. Image super-resolution with cross-scale non-local attention and exhaustive self-exemplars mining[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 5690-5699.
[20] 廉炜雯, 吴斌, 张红英, 等. 高效二阶注意力对偶回归网络的超分辨率重建[J]. 计算机工程与应用, 2022, 58(20): 220-228.
LIAN W W, WU B, ZHANG H Y, et al. Super-resolution reconstruction of efficient second-order attention dual regression network[J]. Computer Engineering and Applications, 2022, 58(20): 220-228.
[21] DING L, TANG H, BRUZZONE L. LANet: local attention embedding to improve the semantic segmentation of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(1): 426-435.
[22] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 770-778.
[23] YANG Y, NEWSAM S. Bag-of-visual-words and spatial extensions for land-use classification[C]//Proceedings of the 18th SIGSPATIAL International Conference on Advances in Geographic Information Systems. New York: ACM, 2010: 270-279.
[24] LONG Y, XIA G S, LI S, et al. On creating benchmark dataset for aerial image interpretation: reviews, guidances, and million-aid[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 4205-4230.
[25] PONOMARENKO N, IEREMEIEV O, LUKIN V, et al. Modified image visual quality metrics for contrast change and mean shift accounting[C]//Proceedings of the 2011 11th International Conference The Experience of Designing and Application of CAD Systems in Microelectronics. Piscataway: IEEE, 2011: 305-311.
[26] WANG Z, LI Q. Information content weighting for perceptual image quality assessment[J]. IEEE Transactions on Image Processing, 2010, 20(5): 1185-1198.
[27] SELVARAJU R R, COGSWELL M, DAS A, et al. Grad-CAM: visual explanations from deep networks via gradient-based localization[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2017: 618-626.