UAV Remote Sensing Object Detection Based on 3D Multi-layer Feature Collaboration

doi:10.3778/j.issn.1673-9418.2401007

Abstract

Abstract: To solve the large proportion of small targets and complex background in UAV (unmanned aerial vehicle) aerial image, the current object detection model has the problems of low accuracy and missed detection of small targets. Based on the YOLOv8s model, this paper proposes a 3D multi-layer feature collaboration UAV remote sensing object detection algorithm. Firstly, based on the coordinate attention, this paper proposes 3D multi-branch coordinate attention (MBCA), which improves the global feature extraction ability of the model and reduces the computation of spatial dimension by increasing the information interaction of channel dimension and the splitting and fusion of extended branches. Secondly, SPD-Conv is used to replace part of the standard convolution, which effectively retains more feature information and speeds up inference during downsampling. Then, a more efficient FastDBB_Bottleneck module is used in the C2f module, combining PConv and DBB structure reparameterization superposition to further reduce the calculation of the model. Finally, PG-Detect detection head is introduced to significantly reduce the calculation and effectively reduce the missed detection rate of small targets. Experimental results on the VisDrone2019 dataset show that the mAP50 value of the proposed method reaches 44.5%, which is 5.7 percentage points higher than that of the YOLOv8s baseline model. Simultaneously, the crack detection verification experiment is carried out on the self-built dam crack dataset, and the mAP50 value of the improved method is 3.3 percentage points higher than that of YOLOv8s, the FPS reaches 289 frames. Experimental results show that the proposed method improves the accuracy and real-time performance of the detection model in complex scene object detection, and has good adaptability and robustness.

Key words: UAV remote sensing, 3D multi-branch coordinate attention (MBCA), YOLOv8, multi-layer feature fusion, small target detection

摘要： 针对无人机航拍图像小目标占比大和背景复杂的特点，当前目标检测模型存在精度低和小目标漏检等问题。基于YOLOv8s模型，提出了三维多层次特征协同的无人机遥感目标检测算法。首先，在坐标注意力的基础上提出了三维多分支坐标注意力（MBCA），通过增加通道维度的信息交互和扩展分支的拆分融合，减少空间维度的计算量，提高了模型全局特征提取能力。其次，采用SPD-Conv替换部分标准卷积，在下采样时有效保留更多特征信息并加快推理速度。然后，在C2f模块中采用了更高效的FastDBB_Bottleneck模块，结合PConv与DBB结构重参数化叠加，以进一步降低模型计算量。最终，通过引入PG-Detect检测头，显著减少计算量并有效降低小目标的漏检率。在VisDrone2019数据集上的实验结果显示，该方法的mAP50值达到了44.5%，较YOLOv8s基线模型提升了5.7个百分点。同时，在自建水坝裂缝数据集上，进行裂缝检测验证实验，改进方法的mAP50值相比YOLOv8s提升了3.3个百分点，FPS达到289帧。实验结果表明在复杂场景目标检测中，所提方法提升了检测模型的精度和实时性，具有良好的适应性和鲁棒性。

关键词: 无人机遥感, 三维多分支坐标注意力（MBCA）, YOLOv8, 多层次特征融合, 小目标检测

LYU Fu, FU Yuheng, HE Lina, YANG Dongpeng. UAV Remote Sensing Object Detection Based on 3D Multi-layer Feature Collaboration[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(5): 1301-1317.

吕伏, 傅宇恒, 贺丽娜, 杨冬鹏. 三维多层次特征协同的无人机遥感目标检测算法[J]. 计算机科学与探索, 2024, 18(5): 1301-1317.

References

[1] MA X, ZHANG Y, ZHANG W, et al. SDWBF algorithm: a novel pedestrian detection algorithm in the aerial scene[J]. Drones, 2022, 6(3): 76.
[2] ZHU P, WEN L, DU D, et al. Detection and tracking meet drones challenge[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2021, 44(11): 7380-7399.
[3] HU Q, LI L, DUAN J, et al. Object detection algorithm of UAV aerial photography image based on anchor-free algorithms[J]. Electronics, 2023, 12(6): 1339.
[4] SHEN S, ZHANG X, YAN W, et al. An improved UAV target detection algorithm based on ASFF-YOLOv5s[J]. Mathematical Biosciences and Engineering, 2023, 20(6): 10773-10789.
[5] LI H, DONG Y, LIU Y, et al. Design and implementation of UAVs for bird’s nest inspection on transmission lines based on deep learning[J]. Drones, 2022, 6(9): 252.
[6] HONG T, YANG Q, WANG P, et al. Multitarget real-time tracking algorithm for UAV IoT[J]. Wireless Communications and Mobile Computing, 2021. DOI:10.1155/2021/9999596.
[7] LIANG F T, ZHOU Y, CHEN X, et al. Review of target detection technology based on deep learning[C]//Proceedings of the 5th International Conference on Control Engineering and Artificial Intelligence. New York: ACM, 2021: 132-135.
[8] CAI Z, VASCONCELOS N. Cascade R-CNN: high quality object detection and instance segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 43(5): 1483-1498.
[9] XIN F, ZHANG H, PAN H. Hybrid dilated multilayer Faster RCNN for object detection[J]. The Visual Computer, 2024,40(1): 393-406.
[10] SHARMA V K, MIR R N. Saliency guided Faster-RCNN (SGFr-RCNN) model for object detection and recognition[J]. Journal of King Saud University (Computer and Information Sciences), 2022, 34(5): 1687-1699.
[11] GIRSHICK R. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2015: 1440-1448.
[12] ZHANG K H, SHEN H K. Solder joint defect detection in the connectors using improved Faster-RCNN algorithm[J]. Applied Sciences, 2021, 11(2): 576.
[13] YANG A M, JIANG T Y, HAN Y, et al. Research on application of on-line melting in-SITU visual inspection of iron ore powder based on Faster R-CNN[J]. Alexandria Engineering Journal, 2022, 61(11): 8963-8971.
[14] KUMAR A, MANIKANDAN R. Brain tumor detection using deep neural network-based classifier[C]//Proceedings of the 2022 International Conference on Innovative Computing and Communications. Singapore: Springer, 2022: 173-181.
[15] GUO F, QIAN Y, RIZOS D, et al. Automatic rail surface defects inspection based on Mask R-CNN[J]. Transportation Research Record, 2021, 2675(11): 655-668.
[16] HE K M, GKIOXARI G, DOLLáR P, et al. Mask R-CNN[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2980-2988.
[17] REN J S, WANG Y. Overview of object detection algorithms using convolutional neural networks[J]. Journal of Computer and Communications, 2022, 10(1): 115-132.
[18] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multi-box detector[C]//Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 21-37.
[19] 李娟莉, 魏代良, 李博, 等. 基于深度学习轻量化的改进SSD煤矸快速分选模型[J]. 东北大学学报(自然科学版), 2023, 44(10): 1474-1480.
LI J L, WEI D L, LI B, et al. Improved SSD rapid separation model of coal gangue based on deep learning and light-weighting[J]. Journal of Northeastern University (Natural Science), 2023, 44(10): 1474-1480.
[20] JIANG P Y, ERGU D, LIU F Y, et al. A review of YOLO algorithm developments[J]. Procedia Computer Science, 2022, 199: 1066-1073.
[21] ZHU X K, LYU S C, WANG X, et al. TPH-YOLOv5: impro-ved YOLOv5 based on transformer prediction head for object detection on drone-captured scenarios[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 11-18, 2021. Piscataway: IEEE, 2021: 2778-2788.
[22] 王鹏飞, 黄汉明, 王梦琪. 改进 YOLOv5 的复杂道路目标检测算法[J]. 计算机工程与应用, 2022, 58(17): 81-92.
WANG P F, HUANG H M, WANG M Q. Complex road target detection algorithm based on improved YOLOv5[J]. Computer Engineering and Applications, 2022, 58(17): 81-92.
[23] 宋怀波, 马宝玲, 尚钰莹, 等. 基于YOLO v7-ECA模型的苹果幼果检测[J]. 农业机械学报, 2023, 54(6): 233-242.
SONG H B, MA B L, SHANG Y Y, et al. Detection of young apple fruits based on the YOLO v7-ECA model[J]. Journal of Agricultural Machinery, 2023, 54(6): 233-242.
[24] 陈占龙, 李双江, 徐永洋, 等. 高分影像密集建筑物Correg-YOLOv3检测方法[J]. 测绘学报, 2022, 51(12): 2531-2540.
CHEN Z L, LI S J, XU Y Y, et al. Dense building detection in high-resolution remote sensing imagery using Correg-YOLOv3[J]. Acta Geodaetica et Cartographica Sinica, 2022, 51(12): 2531-2540.
[25] 苏俊楷, 段先华, 叶赵兵. 改进YOLOv5算法的玉米病害检测研究[J]. 计算机科学与探索, 2023, 17(4): 933-941.
SU J K, DUAN X H, YE Z B. Research on corn disease detection based on improved YOLOv5 algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(4): 933-941.
[26] ZHANG J, WAN G, JIANG M, et al. Small object detection in UAV image based on improved YOLOv5[J]. Systems Science & Control Engineering, 2023, 11(1): 2247082.
[27] 徐光达, 毛国君. 多层级特征融合的无人机航拍图像目标检测[J]. 计算机科学与探索, 2023, 17(3): 635-645.
XU G D, MAO G J. Aerial image object detection of UAV based on multi-level feature fusion[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(3): 635-645.
[28] 李安达, 吴瑞明, 李旭东. 改进YOLOv7的小目标检测算法研究[J]. 计算机工程与应用, 2024, 60(1): 122-134.
LI A D, WU R M, LI X D. Research on improving YOLOv7’s small target detection algorithm[J]. Computer Engineering and Applications, 2024, 60(1): 122-134.
[29] 王春梅, 刘欢. YOLOv8-VSC: 一种轻量级的带钢表面缺陷检测算法[J]. 计算机科学与探索, 2024, 18(1): 151-160.
WANG C M, LIU H. YOLOv8-VSC: lightweight algorithm for strip surface defect detection[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(1): 151-160.
[30] WANG G, CHEN Y, AN P, et al. UAV-YOLOv8: a small-object-detection model based on improved YOLOV8 for UAV aerial photography scenarios[J]. Sensors, 2023, 23(16): 7190.
[31] 吴建成, 郭荣佐, 成嘉伟, 等. 注意力特征融合的快速遥感图像目标检测算法[J]. 计算机工程与应用, 2024, 60(1): 207-216.
WU J C, GUO R Z, CHENG J W, et al. Fast remote sensing image object detection algorithm based on attention feature fusion[J]. Computer Engineering and Applications, 2024, 60(1): 207-216.
[32] WOO S, PARK J, LEE J Y, et al. CBAM: convolutional block attention module[C]//Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 3-19.
[33] HOU Q, ZHOU D, FENG J. Coordinate attention for efficient mobile network design[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 13713-13722.
[34] BOUREAU Y L, PONCE J, LECUN Y. A theoretical analysis of feature pooling in visual recognition[C]//Proceedings of the 27th International Conference on Machine Learning, Haifa, Jun 21-24, 2010: 111-118.
[35] SUNKARA R, LUO T. No more strided convolutions or pooling: a new CNN building block for low-resolution images and small objects[C]//Proceedings of the 2022 Joint European Conference on Machine Learning and Knowledge Discovery in Databases. Cham: Springer, 2022: 443-459.
[36] CHEN J, KAO S H, HE H, et al. Run, don??t walk: chasing higher FLOPs for faster neural networks[C]//Proceedings of the 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2023: 12021-12031.
[37] DING X, ZHANG X, HAN J, et al. Diverse branch block: building a convolution as an inception-like unit[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition. Piscataway: IEEE, 2021: 10886-10895.
[38] BRODIE C, CONSTANTIN A, LUKAS A, et al. Flops for complete intersection Calabi-Yau threefolds[J]. Journal of Geometry and Physics, 2023, 186: 104767.
[39] LEE Y, PARK J, LEE C O. Two-level group convolution[J]. Neural Networks, 2022, 154: 323-332.
[40] DU D, ZHU P, WEN L, et al. VisDrone-DET2019: the vision meets drone object detection in image challenge results[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision. Piscataway: IEEE, 2019: 213-226.