Remote Sensing Image Object Detection Algorithm Based on Multi-branch Feature Mapping

doi:10.3778/j.issn.1673-9418.2305090

Abstract

Abstract: Due to the complex background, small and dense targets, and large scale continuous changes in remote sensing images, universal object detectors are difficult to adapt well, resulting in poor detection performance. To address the above issues, a multi-branch feature mapping based remote sensing image object detection algorithm is proposed based on the YOLOv5s model. Firstly, a RepVGG module combined with gated channel transformation is designed using structural reparameterization technology. Its series structure is used to replace the C3 module of the original backbone network, aggregating global contextual information and enhancing feature expression and extraction capabilities. Secondly, the adaptive exponential weighted pooling method and the sampling method of inverse process reconstruction feature fusion network are used to maximize the retention of feature information and improve the detection performance of smaller targets. Finally, the combination of EIoU and Focal Loss is introduced as the loss function of the model to optimize the regression rate of the prediction box and the loss weight distribution of difficult and easy samples, further improving the positioning accuracy. The experimental results on the DIOR and NWPU VHR-10 datasets show that the average accuracy of the proposed algorithm reaches 92.2% and 92.5%, respectively, which is 3.5 percentage points and 5.6 percentage points higher than YOLOv5s, achieving better detection performance. At the same time, the real-time performance also meets the requirements of remote sensing image object detection in actual scenes.

Key words: remote sensing images, structural reparameterization, gated channel transformation, sampling method, loss weight allocation

摘要： 由于遥感图像具有背景复杂、目标较小且密集、尺度连续变化大等特点，通用目标检测器难以较好地适应，导致检测效果不佳。针对以上问题，基于YOLOv5s模型，提出一种多分支特征映射的遥感图像目标检测算法。首先，利用结构重参数化技术设计一种结合门控通道转换的RepVGG模块，采用其串联结构替换原主干网络的C3模块，聚合全局上下文信息，增强特征表达和特征提取能力；其次，使用自适应指数加权池化方法以及逆过程重构特征融合网络的采样方式，最大化地保留特征信息，改善较小目标的检测效果；最后，引入EIoU和Focal Loss组合作为模型的损失函数，优化预测框的回归速率以及难易样本的损失权重分配，进一步提高定位精度。在DIOR和NWPU VHR-10数据集上的实验结果表明，提出算法的平均精度均值分别达到92.2%、92.5%，较YOLOv5s分别提高了3.5个百分点、5.6个百分点，达到了更好的检测效果，同时实时性也满足实际场景下的遥感图像目标检测。

关键词: 遥感图像, 结构重参数化, 门控通道转换, 采样方式, 损失权重分配

MIN Feng, KUANG Yonggang, HAO Linlin, PENG Weiming. Remote Sensing Image Object Detection Algorithm Based on Multi-branch Feature Mapping[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(6): 1543-1555.

闵锋, 况永刚, 郝琳琳, 彭伟明. 多分支特征映射的遥感图像目标检测算法[J]. 计算机科学与探索, 2024, 18(6): 1543-1555.

References

[1] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2005: 886-893.
[2] HAREL J, KOCH C, PERONA P. Graph-based visual saliency[C]//Advances in Neural Information Processing Systems 19, Vancouver, Dec 4-7, 2006. Cambridge: MIT Press, 2007: 545-552.
[3] SEDAGHAT A, EBADI H. Remote sensing image matching based on adaptive binning SIFT descriptor[J]. IEEE Transactions on Geoscience and Remote Sensing, 2015, 53(10): 5283-5293.
[4] GIRSHICK R, DONAHUE J, DARRELL T, et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition. Washington:IEEE Computer Society, 2014: 580-587.
[5] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Washington:IEEE Computer Society, 2015: 1440-1448.
[6] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]// Advances in Neural Information Processing Systems 28, Montreal, Dec 7-12, 2015: 91-99.
[7] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2016: 779-788.
[8] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//Proceedings of the 14th European Conference on Computer Vision. Cham: Springer, 2016: 21-37.
[9] 苏树智, 谢玉麒. 基于距离约束的改进FCOS遥感图像检测方法[J]. 计算机工程与应用, 2023, 59(10): 227-235.
SU S Z, XIE Y Q. Improved FCOS remote sensing image detection method based on distance constraint[J]. Computer Engineering and Applications, 2023, 59(10): 227-235.
[10] 张云佐, 郭威, 蔡昭权, 等. 联合多尺度与注意力机制的遥感图像目标检测[J]. 浙江大学学报(工学版), 2022, 56(11): 2215-2223.
ZHANG Y Z, GUO W, CAI Z Q, et al. Remote sensing image target detection combining multi-scale and attention mechanism[J]. Journal of Zhejiang University (Engineering Edition), 2022, 56(11): 2215-2223.
[11] TAN M, PANG R, LE Q V. EfficientDet：scalable and efficient object detection[C]//Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2020: 1-10.
[12] 赵文清, 康怿瑾, 赵振兵, 等. 改进YOLOv5s的遥感图像目标检测[J]. 智能系统学报, 2023, 18(1): 86-95.
ZHAO W Q, KANG Y J, ZHAO Z B, et al. A remote sensing image object detection algorithm with improved YOLOv5s[J]. CAAI Transactions on Intelligent Systems, 2023, 18(1): 86-95.
[13] WANG Q, WU B, ZHU P, et al. ECA-Net: efficient channel attention for deep convolutional neural networks[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11534-11542.
[14] LIU Z, LIN Y, CAO Y, et al. Swin transformer: hierarchical vision transformer using shifted windows[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2021: 10012-10022.
[15] HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 13713-13722.
[16] XU X, FENG Z, CAO C, et al. An improved swin transformer-based model for remote sensing object detection and instance segmentation[J]. Remote Sensing, 2021, 13(23): 4779.
[17] YE Y, REN X, ZHU B, et al. An adaptive attention fusion mechanism convolutional network for object detection in remote sensing images[J]. Remote Sensing, 2022, 14(3): 516.
[18] 李婕, 周顺, 朱鑫潮, 等. 结合多通道注意力的遥感图像飞机目标检测[J]. 计算机工程与应用, 2022, 58(1): 209-217.
LI J, ZHOU S, ZHU X C, et al. Remote sensing image aircraft target detection combined with multiple channel attention[J]. Computer Engineering and Applications, 2022, 58(1): 209-217.
[19] DING X, ZHANG X, MA N, et al. RepVGG: making VGG-style convnets great again[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 13733-13742.
[20] ZHANG Y F, REN W, ZHANG Z, et al. Focal and efficient IOU loss for accurate bounding box regression[J]. Neurocomputing, 2022, 506: 146-157.
[21] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2017: 2980-2988.
[22] YANG Z, ZHU L, WU Y, et al. Gated channel transformation for visual recognition[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2020: 11794-11803.
[23] WANG Z, BOVIK A C, SHEIKH H R, et al. Image quality assessment: from error visibility to structural similarity[J]. IEEE Transactions on Image Processing, 2004, 13(4): 600-612.
[24] ZHAI S, WU H, KUMAR A, et al. S3Pool: pooling with stochastic spatial sampling[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition. Washington: IEEE Computer Society, 2017: 4970-4978.
[25] STERGIOU A, POPPE R. AdaPool: exponential adaptive pooling for information-retaining downsampling[J]. IEEE Transactions on Image Processing, 2022, 32: 251-266.
[26] REZATOFIGHI H, TSOI N, GWAK J Y, et al. Generalized intersection over union: a metric and a loss for bounding box regression[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition.Piscataway: IEEE, 2019: 658-666.
[27] TIAN Z, SHEN C, CHEN H, et al. FCOS: fully convolutional one-stage object detection[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 9627-9636.
[28] CARION N, MASSA F, SYNNAEVE G, et al. End-to-end object detection with transformers[C]//Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 213-229.
[29] LONG X, DENG K, WANG G, et al. PP-YOLO: an effective and efficient implementation of object detector[J]. arXiv:2007.12099, 2020.
[30] CHEN T, LI R, FU J, et al. Tucker bilinear attention network for multi-scale remote sensing object detection[J]. arXiv:2303.05329, 2023.
[31] ZHOU L, WEI H, LI H, et al. Arbitrary-oriented object detection in remote sensing images based on polar coordinates[J]. IEEE Access, 2020, 8: 223373-223384.
[32] GE Z, LIU S, WANG F, et al. YOLOx: exceeding YOLO series in 2021[J]. arXiv:2107.08430, 2021.
[33] HUANG X, WANG X, LV W, et al. PP-YOLOv2: a practical object detector[J]. arXiv:2104.10419, 2021.
[34] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.
[35] SHI L, LI Y, ZHU X. Anchor free remote sensing detector based on solving discrete polar coordinate equation[J]. arXiv:2303.11694, 2023.
[36] 曾伦杰, 储珺, 陈昭俊. 二阶段锚框和类均衡损失的遥感图像目标检测[J]. 图学学报, 2023, 44(2): 249-259.
ZENG L J, CHU J, CHEN Z J. Object detection in remote sensing image based on two-stage anchor and class balanced loss[J]. Journal of Graphics, 2023, 44(2): 249-259.
[37] ZEILER M D, FERGUS R. Stochastic pooling for regularization of deep convolutional neural networks[J]. arXiv:1301.3557, 2013.
[38] SAEEDAN F, WEBER N, GOESELE M, et al. Detail-preserving pooling in deep networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Reco-gnition. Washington: IEEE Computer Society, 2018: 9108-9116.
[39] GAO Z, WANG L, WU G. LIP: local importance-based pooling[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 3355-3364.