结合Graph-FPN与稳健优化的开放世界目标检测

doi:10.3778/j.issn.1673-9418.2211068

摘要/Abstract

摘要： 开放世界目标检测（OWOD）要求检测图像中所有已知和未知的目标类别，同时模型必须逐步学习新的类别以自适应更新知识。针对ORE方法存在的未知目标召回率低以及增量学习的灾难性遗忘等问题，提出一种基于图特征金字塔的稳健优化开放世界目标检测方法（GARO-ORE）。首先，利用Graph-FPN中的超像素图像结构以及上下文层和层次层的分层设计，获取丰富的语义信息并帮助模型准确定位未知目标；之后，利用稳健优化方法对不确定性综合考量，提出了基于平坦极小值的基类学习策略，极大限度地保证模型在学习新类别的同时避免遗忘先前学习到的类别知识；最后，采用基于知识迁移的新增类别权值初始化方法提高模型对新类别的适应性。在OWOD数据集上的实验结果表明，GARO-ORE在未知类别召回率上取得较优的检测结果，在10+10、15+5、19+1三种增量目标检测（iOD）任务中，其mAP指标分别提升了1.38、1.42和1.44个百分点。可以看出，GARO-ORE能够较好地提高未知目标检测的召回率，并且在有效缓解旧任务灾难性遗忘问题的同时促进后续任务的学习。

关键词: 开放世界目标检测（OWOD）, 图特征金字塔网络, 平坦极小值, 知识迁移

Abstract: Open world object detection (OWOD) requires detecting all known and unknown object categories in the image, and the model must gradually learn new categories to adaptively update knowledge. Aiming at the problems of low recall rate of unknown objects and catastrophic forgetting of incremental learning in ORE (open world object detection) method, this paper proposes adjustable robust optimization of ORE based on graph feature pyramid (GARO-ORE). Firstly, using the superpixel image structure in Graph-FPN and the hierarchical design of context layer and hierarchical layer, rich semantic information can be obtained and the model can accurately locate unknown object. Then, using the robust optimization method to comprehensively consider the uncertainty, a base class learning strategy based on flat minimum is proposed, which greatly ensures that the model avoids forgetting the previously learnt category knowledge while learning new categories. Finally, the classification weights initiali-zation method based on knowledge transfer is used to improve the adaptability of the model to new classes. Experimental results on the OWOD dataset show that GARO-ORE achieves better detection results on the recall rate of unknown categories. In the three types of incremental object detection tasks of 10 + 10, 15 + 5, and 19 + 1, the mAP is increased by 1.38, 1.42 and 1.44 percentage points, respectively. It can be seen that GARO-ORE can improve the recall rate of unknown object detection, and promote the learning of subsequent tasks while effectively alleviating the catastrophic forgetting problem of old tasks.

Key words: open world object detection (OWOD), graph feature pyramid network, flat minimum, knowledge transfer

谢斌红, 张鹏举, 张睿. 结合Graph-FPN与稳健优化的开放世界目标检测[J]. 计算机科学与探索, 2023, 17(12): 2954-2966.

XIE Binhong, ZHANG Pengju, ZHANG Rui. Open World Object Detection Combining Graph-FPN and Robust Optimization[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(12): 2954-2966.

参考文献

[1] 蒋弘毅, 王永娟, 康锦煜. 目标检测模型及其优化方法综述[J]. 自动化学报, 2021, 47(6): 1232-1255.
JIANG H Y, WANG Y J, KANG J Y. A survey of object detection models and its optimization methods[J]. Acta Automatica Sinica, 2021, 47(6): 1232-1255.
[2] GIRSHICK R B. Fast R-CNN[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society， 2015: 1440-1448.
[3] HE K, GKIOXARI G, DOLLAR P, et al. Mask R-CNN[J]. IEEE Transactions on Pattern Analysis and Machine Intel-ligence, 2020, 42(2): 386-397.
[4] SINGH B, NAJIBI M, DAVIS L S. SNIPER: efficient multi-scale training[C]//Advances in Neural Information Proces-sing Systems 31, Montréal, Dec?3-8,?2018: 9333-9343.
[5] NIE J, ANWER R M, CHOLAKKAL H, et al. Enriched feature guided refinement network for object detection[C]// Proceedings of the 2019 IEEE/CVF International Confe-rence on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 9536-9545.
[6] PANG Y, WANG T, ANWER R M, et al. Efficient featu-rized image pyramid network for single shot detector[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 7336-7344.
[7] BOCHKOVSKIY A, WANG C Y, LIAO H Y M. YOLOv4: optimal speed and accuracy of object detection[J]. arXiv:2004.10934, 2020.
[8] 李青援, 邓赵红, 罗晓清, 等. 注意力与跨尺度融合的SSD目标检测算法[J]. 计算机科学与探索, 2022, 16(11): 2575-2586.
LI Q Y, DENG Z H, LUO X Q, et al. SSD object detection algorithm with attention and cross-scale fusion[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2575-2586.
[9] JOSEPH K J, KHAN S, KHAN F S, et al. Towards open world object detection[C]//Proceedings of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Jun 19-25, 2021. Piscataway: IEEE, 2021: 5830-5840.
[10] WANG X, HUANG T E, DARRELl T, et al. Frustratingly simple few-shot object detection[J]. arXiv:2003.06957, 2020.
[11] HALL D, DAYOUB F, SKINNER J, et al. Probabilistic object detection: definition and evaluation[C]//Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, Mar 1-5, 2020. Pisca-taway: IEEE, 2020: 1020-1029.
[12] 高菲, 杨柳, 李晖. 开放集识别研究综述[J]. 南京大学学报 (自然科学版), 2022, 58(1): 115-134.
GAO F, YANG L, LI H. A survey on open set recognition[J]. Journal of Nanjing University (Natural Science), 2022, 58(1): 115-134.
[13] MILLER D, DAYOUB F, MILFORD M, et al. Evaluating merging strategies for sampling-based uncertainty techni-ques in object detection[C]//Proceedings of the 2019 Inter-national Conference on Robotics and Automation,Montreal, May 20-24, 2019. Piscataway: IEEE, 2019: 2348-2354.
[14] DHAMIJA A, GUNTHER M, VENTURA J, et al. The overlooked elephant of object detection: open set[C]//Proce-edings of the 2020 IEEE Winter Conference on Applic-ations of Computer Vision, Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 1010-1019.
[15] PERERA P, MORARIU V I, JAIN R, et al. Generative-discriminative feature representations for open-set recog-nition[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 11811-11820.
[16] YOSHIHASHI R, SHAO W, KAWAKAMI R, et al. Class-ification-reconstruction learning for open-set recognition[C]//Proceedings of the 2019 IEEE Conference on Com-puter Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4016-4025.
[17] BENDALE A, BOULT T. Towards open world recognition[C]//Proceedings of the 2015 IEEE Conference on Com-puter Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 1893-1902.
[18] PENG C, ZHAO K, LOVELL B C. Faster ILOD: incre-mental learning for object detectors based on faster RCNN[J]. Pattern Recognition Letters, 2020, 140: 109-115.
[19] SHMELKOV K, SCHMID C, ALAHARI K. Incremental learning of object detectors without catastrophic forgetting[C]//Proceedings of the 2017 IEEE International Confe-rence on Computer Vision, Venice, Oct 22-29, 2017. Wash-ington: IEEE Computer Society, 2017: 3420-3429.
[20] 刘冰瑶, 刘进锋. 增量学习研究综述[J]. 现代计算机, 2022, 28(13): 72-75.
LIU B Y, LIU J F. Literature review of incremental learning[J]. Modern Computer, 2022, 28(13): 72-75.
[21] REBUFFI S A, KOLESNIKOV A, SPERL G, et al. iCaRL: incremental classifier and representation learning[C]//Proc-eedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Wash-ington: IEEE Computer Society, 2017: 5533-5542.
[22] KANAKIS M, BRUGGEMANN D, SAHA S, et al. Repar-ameterizing convolutions for incremental multi-task learning without task interference[C]//LNCS 12365: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 689-707.
[23] KIRKPATRICK J, PASCANU R, RABINOWITZ N, et al. Overcoming catastrophic forgetting in neural networks[J]. Proceedings of the National Academy of Sciences, 2017, 114(13): 3521-3526.
[24] ZHAO G M, GE W F, YU Y Z. GraphFPN: graph feature pyramid network for object detection[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 2743-2752.
[25] MANINIS K K, PONT-TUSET J, ARBELáEZ P, et al. Convolutional oriented boundaries: from image segme-ntation to high-level tasks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 819-833.
[26] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al.Graph attention networks[J]. arXiv:1710.10903, 2017.
[27] CHAUDHARI P, CHOROMANSKA A, SOATTO S, et al. Entropy-SGD: biasing gradient descent into wide valleys[J]. Journal of Statistical Mechanics: Theory and Exper-iment, 2019(12): 124018.
[28] SHI G, CHEN J, ZHANG W, et al. Overcoming cata-strophic forgetting in incremental few-shot learning by finding flat minima[C]//Advances in Neural Information Processing Systems?34,?Dec?6-14,?2021: 6747-6761.
[29] DU J, ZHOU D, FENG J, et al. Sharpness-aware training for free[C]//Advances in Neural nformation Processing Sys-tems 35, New Orleans, Nov 28-Dec 9, 2022: 23439-23451.
[30] DONG N, ZHANG Y, DING M, et al. Open world DETR: transformer based open world object detection[J]. arXiv:2212.02969, 2022.