面向遥感图像检索的自适应样本类型判别研究

doi:10.3778/j.issn.1673-9418.2402031

摘要/Abstract

摘要： 遥感图像内容复杂，类别丰富，存在较多难以判别的图像，导致遥感图像检索性能不佳。为此，提出自适应样本类型判别方法（ASTD），将样本类型动态地分为简单样本、普通样本和困难样本，网络依据样本的类型进行不同程度的学习，从而有效提高特征的判别能力。设计了一个SHash网络，该网络以Swin Transformer为骨干，在网络的最后加上哈希层，该网络能够在全局上捕获图像的语义信息，提高特征的表达能力和检索效率；为了让同一类别图像更加聚集，并更好地区分不同类别的图像，给每个类别定义一个哈希中心，规定输入样本自身类别所对应的中心为该样本的正中心，其他中心为该样本的负中心；提出样本类型判别损失STDLoss，根据样本与正负中心的距离关系自适应判别样本的类型，从而提高网络对各类型样本的学习能力。在UC-Merced和AID两个遥感数据集上与DSH、CSQ、SHC等五种哈希方法进行了比较，实验结果表明，基于ASTD方法训练的网络可以更好地学习样本的特征，提高检索性能。

关键词: 遥感图像检索, 样本类型, Swin Transformer, 哈希, 自适应判别

Abstract: Remote sensing images are complex in content and rich in categories, and there are numerous images difficult in discrimination, resulting in poor performance of remote sensing image retrieval. For this reason, the adaptive sample type discrimination (ASTD) method is proposed, which dynamically categorizes the sample types into simple samples, ordinary samples and difficult samples, and the network performs different degrees of learning based on the types of samples, so as to effectively improve the discriminative ability of features. Firstly, an SHash network is designed, which takes Swin Transformer as the backbone and adds a hash layer at the end of the network. This network captures the semantic information of images globally, improving feature representation and retrieval efficiency. Secondly, in order to make the same category of images more aggregated, and to better distinguish different categories of images, a hash center is defined for each category. The center corresponding to the input sample’s own category is specified as the positive center of the sample, and the other centers are the negative centers of the sample. Finally, the sample type discriminative loss STDLoss is proposed to adaptively discriminate the type of samples based on distance relationship between samples and positive and negative centers, so as to improve the network’s ability to learn from each type of samples. Comparison with five hashing methods such as DSH, CSQ and SHC on two remote sensing datasets, UC-Merced and AID, experimental results show that the network trained based on the ASTD can better learn the features of the samples and improve the retrieval performance.

Key words: remote sensing image retrieval, sample type, Swin Transformer, hash, adaptive discrimination

邵徽虎, 葛芸, 熊俊杰, 余洁洁. 面向遥感图像检索的自适应样本类型判别研究[J]. 计算机科学与探索, 2024, 18(11): 2996-3005.

SHAO Huihu, GE Yun, XIONG Junjie, YU Jiejie. Research on Adaptive Sample Type Discrimination for Remote Sensing Image Retrieval[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(11): 2996-3005.

参考文献

[1] WANG H, CHEN X Z, ZHANG T X, et al. CCTNet: coupled CNN and transformer network for crop segmentation of remote sensing images[J]. Remote Sensing, 2022, 14(9): 1956.
[2] 王浩桐, 郭中华. 锚框策略匹配的SSD飞机遥感图像目标检测[J]. 计算机科学与探索, 2022, 16(11): 2596-2608.
WANG H T, GUO Z H. Target detection of SSD aircraft remote sensing images based on anchor frame strategy matching[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2596-2608.
[3] SHAO Z F, ZHOU W X, DENG X Q, et al. Multilabel remote sensing image retrieval based on fully convolutional network[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2020, 13: 318-328.
[4] YE F M, WU K L, ZHANG R G, et al. Multi-scale feature fusion based on PVTv2 for deep hash remote sensing image retrieval[J]. Remote Sensing, 2023, 15(19): 4729.
[5] LIU Z, LIN Y T, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 9992-10002.
[6] TOUVRON H, CORD M, DOUZE M, et al. Training data-efficient image transformers & distillation through attention[C]//Proceedings of the 38th International Conference on Machine Learning, Jul 18-24, 2021: 10347-10357.
[7] 苗壮, 赵昕昕, 李阳, 等. 基于Swin Transformer的深度有监督哈希图像检索方法[J]. 湖南大学学报(自然科学版), 2023, 50(8): 62-71.
MIAO Z, ZHAO X X, LI Y, et al. Deep supervised hashing image retrieval method based on Swin Transformer[J]. Journal of Hunan University (Natural Sciences), 2023, 50(8): 62-71.
[8] YANG Y, NEWSAM S. Geographic image retrieval using local invariant features[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013, 51(2): 818-832.
[9] XIA G S, HU J W, HU F, et al. AID: a benchmark data set for performance evaluation of aerial scene classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2017, 55(7): 3965-3981.
[10] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, Long Beach, Dec 4-9, 2017: 5998-6008.
[11] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An image is worth 16×16 words: transformers for image recognition at scale[C]//Proceedings of the 9th International Conference on Learning Representations, Vienna, May 3-7, 2021.
[12] RAHHAL M M A, BENCHERIF M A, BAZI Y, et al. Contrasting dual transformer architectures for multi-modal remote sensing image retrieval[J]. Applied Sciences, 2022, 13(1): 282.
[13] AL RAHHAL M M, BAZI Y, ALSHARIF N A, et al. Multilanguage transformer for improved text to remote sensing image retrieval[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2022, 15: 9115-9126.
[14] CHEN Y X, WANG F, LU L, et al. Unsupervised transformer balanced hashing for multispectral remote sensing image retrieval[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2023, 16: 7089-7099.
[15] GIONIS A, INDYK P, MOTWANI R. Similarity search in high dimensions via hashing[C]//Proceedings of the 25th International Conference on Very Large Data Bases, Edinburgh, Sep 7-10, 1999. San Francisco: Morgan Kaufmann, 1999: 518-529.
[16] LI P, HAN L R, TAO X W, et al. Hashing nets for hashing: a quantized deep learning to hash framework for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 58(10): 7331-7345.
[17] TAN X Y, ZOU Y, GUO Z Y, et al. Deep contrastive self-supervised hashing for remote sensing image retrieval[J]. Remote Sensing, 2022, 14(15): 3643.
[18] SCHROFF F, KALENICHENKO D, PHILBIN J. FaceNet: a unified embedding for face recognition and clustering[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 815-823.
[19] LIU H M, WANG R P, SHAN S G, et al. Deep supervised hashing for fast image retrieval[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 2064-2072.
[20] SU S P, ZHANG C, HAN K, et al. Greedy hash: towards fast optimization for accurate hash coding in CNN[C]//Advances in Neural Information Processing Systems 31, Montréal, Dec 3-8, 2018: 806-815.
[21] CAO Z J, LONG M S, WANG J M, et al. HashNet: deep learning to hash by continuation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 5608-5617.
[22] YUAN L, WANG T, ZHANG X P, et al. Central similarity quantization for efficient image and video retrieval[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020.Piscataway: IEEE, 2020: 3080-3089.
[23] WANG L D, PAN Y, LAI H J, et al. Image retrieval with well-separated semantic hash centers[C]//Proceedings of the 2022 Asian Conference on Computer Vision, Macao, China, Dec 1-8, 2022. Cham: Springer, 2022: 978-994.
[24] ROY S, SANGINETO E, DEMIR B, et al. Metric-learning-based deep hashing network for content-based retrieval of remote sensing images[J]. IEEE Geoscience and Remote Sensing Letters, 2020, 18(2): 226-230.
[25] WANG Y M, JI S P, LU M, et al. Attention boosted bilinear pooling for remote sensing image retrieval[J]. International Journal of Remote Sensing, 2020, 41(7): 2704-2724.
[26] YE F M, DONG M, LUO W, et al. A new re-ranking method based on convolutional neural network and two image-to-class distances for remote sensing image retrieval[J]. IEEE Access, 2019, 7: 141498-141507.
[27] 叶发茂, 孟祥龙, 董萌, 等. 遥感图像蚁群算法和加权图像到类距离检索法[J]. 测绘学报, 2021, 50(5): 612-620.
YE F M, MENG X L, DONG M, et al. Remote sensing image retrieval with ant colony optimization and a weighted image-to-class distance[J]. Acta Geodaetica et Cartographica Sinica, 2021, 50(5): 612-620.
[28] ZHUO Z, ZHOU Z. Remote sensing image retrieval with Gabor-CA-ResNet and split-based deep feature transform network[J]. Remote Sensing, 2021, 13(5): 869.
[29] ZHANG M D, CHENG Q M, LUO F, et al. A triplet nonlocal neural network with dual-anchor triplet loss for high-resolution remote sensing image retrieval[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2021, 14: 2711-2723.
[30] SONG W W, LI S T, BENEDIKTSSON J A. Deep hashing learning for visual and semantic retrieval of remote sensing images[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(11): 9661-9672.
[31] LIU C, MA J J, TANG X, et al. Deep hash learning for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2020, 59(4): 3420-3443.
[32] SONG W W, GAO Z, DIAN R W, et al. Asymmetric hash code learning for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2022, 60: 1-14.
[33] LI X Y, WEI S, WANG J, et al. Adaptive multi-proxy for remote sensing image retrieval[J]. Remote Sensing, 2022, 14(21): 5615.
[34] HUANG M L, DONG L, DONG W S, et al. Supervised contrastive learning based on fusion of global and local features for remote sensing image retrieval[J]. IEEE Transactions on Geoscience and Remote Sensing, 2023, 61: 1-13.