无监督学习的车辆重识别方法研究综述

doi:10.3778/j.issn.1673-9418.2209100

摘要/Abstract

摘要： 车辆重识别作为智能交通系统的关键技术之一，旨在从不同监控场景下识别同一车辆，对构建平安智慧城市起着重要作用。随着计算机视觉技术的不断发展，使用监督学习的重识别方法存在训练过程对人工标注依赖强、场景泛化能力弱的问题，因此无监督学习的车辆重识别逐渐成为近年来研究的重点。首先，介绍了当前主流的车辆重识别数据集以及常用的模型评价指标。然后，系统梳理了近几年基于无监督学习的车辆重识别方法，根据目前的研究思路将这些方法归纳为生成对抗网络和聚类算法两大类；从域偏差、跨视域偏差以及数据样本信息不足的问题出发，将前者进一步分为风格转换、多视角生成和数据增强三类；又针对标签的问题，将后者分为伪标签的无监督域适应和无需标签信息两类；以解决问题为着手点，总结出每类方法的基本原理、优缺点以及在主流数据集上的性能结果。最后，讨论分析了目前无监督学习的车辆重识别所面临的挑战，并对该研究方向的未来工作进行展望。

关键词: 智能交通, 车辆重识别, 无监督学习, 生成对抗网络, 聚类

Abstract: As one of the key technologies of intelligent transportation systems, vehicle re-identification (Re-ID) aims to retrieve the same vehicle from different monitoring scenes and plays an important role in building a safe and smart city. With the continuous development of computer vision, the Re-ID method of using supervised learning suffers from the problems of strong reliance on manual annotation in the training process and weak scene generalization ability, so unsupervised learning of vehicle Re-ID gradually becomes the focus of research in recent years. Firstly, the present mainstream vehicle Re-ID datasets and the commonly used model evaluation metrics are introduced. Then, latest unsupervised learning-based vehicle Re-ID methods are grouped into two categories: gene-rative adversarial networks and clustering algorithms according to the current research ideas. Starting from the problems of domain deviation, cross-view deviation and insufficient information of data samples, the former is further divided into three categories of style transfer, multi-view generation, and data augmentation. For the labeling pro-blem, the latter can be divided into two categories of pseudo-labeled unsupervised domain adaptation and no label infor-mation required. With problem solving as the starting point, the fundamentals, advantages and disadvantages, and performance results of each type of method on mainstream datasets are summarized. Finally, the challenges faced by the current unsupervised learning for vehicle Re-ID are analyzed, and the future work in this research direction is prospected.

Key words: intelligent transportation, vehicle re-identification, unsupervised learning, generative adversarial networks, clustering

徐岩, 郭晓燕, 荣磊磊. 无监督学习的车辆重识别方法研究综述[J]. 计算机科学与探索, 2023, 17(5): 1017-1037.

XU Yan, GUO Xiaoyan, RONG Leilei. Review of Research on Vehicle Re-identification Methods with Unsupervised Learning[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(5): 1017-1037.

参考文献

[1] HU X, XU X, XIAO Y, et al. SINet: a scale-insensitive convolutional neural network for fast vehicle detection[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 20(3): 1010-1019.
[2] 王左帅, 谭德荣, 徐艺, 等. 一种基于多类特征融合的车辆识别方法[J]. 现代电子技术, 2020, 43(1): 31-34.
WANG Z S, TAN D R, XU Y, et al. A vehicle recognition method based on multi-class feature fusion[J]. Modern Elec-tronics Technique, 2020, 43(1): 31-34.
[3] 杨娟, 曹浩宇, 汪荣贵, 等. 基于语义DCNN特征融合的细粒度车型识别模型[J]. 计算机辅助设计与图形学学报, 2019, 31(1): 141-157.
YANG J, CAO H Y, WANG R G, et al. Fine-grained car recognition model based on semantic DCNN features fusion[J]. Journal of Computer-Aided Design & Computer Grap-hics, 2019, 31(1): 141-157.
[4] LI X, YU L, CHANG D, et al. Dual cross-entropy loss for small-sample fine-grained vehicle classification[J]. IEEE Transactions on Vehicular Technology, 2019, 68(5): 4204-4212.
[5] TAN X, WANG Z, JIANG M, et al. Multi-camera vehicle tracking and re-identification based on visual and spatial-temporal features[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 275-284.
[6] JI X, ZHANG G, CHEN X, et al. Multi-perspective tracking for intelligent vehicle[J]. IEEE Transactions on Intelligent Transportation Systems, 2018, 19(2): 518-529.
[7] ASTHANA S, SHARMA N, SINGH R. Vehicle number plate recognition using multiple layer back propagation neural networks[J]. International Journal of Computer Technology and Electronics Engineering, 2011, 1(1): 35.
[8] PAN M S, YAN J B, XIAO Z H. Vehicle license plate cha-racter segmentation[J]. International Journal of Automation and Computing, 2008, 5(4): 425-432.
[9] MUSA Z B, WATADA J. A grid-computing based multi-camera tracking system for vehicle plate recognition[J]. Ky-bernetika, 2006, 42(4): 495-514.
[10] KWONG K, KAVALER R, RAJAGOPAL R, et al. Arterial travel time estimation based on vehicle re-identification using wireless magnetic sensors[J]. Transportation Research Part C: Emerging Technologies, 2009, 17(6): 586-606.
[11] PRINSLOO J, MALEKIAN R. Accurate vehicle location system using RFID, an Internet of things approach[J]. Sensors, 2016, 16(6): 825.
[12] ZHANG Z, TAN T, HUANG K, et al. Three-dimensional deformable-model-based localization and recognition of road vehicles[J]. IEEE Transactions on Image Processing, 2011, 21(1): 1-13.
[13] LOWE D G. Distinctive image features from scale-invariant keypoints[J]. International Journal of Computer Vision, 2004, 60(2): 91-110.
[14] LIU X, LIU W, MA H, et al. Large-scale vehicle re-identi-fication in urban surveillance videos[C]//Proceedings of the 2016 IEEE International Conference on Multimedia and Expo, Seattle, Jul 11-15, 2016. Washington: IEEE Computer Society, 2016: 1-6.
[15] ZAPLETAL D, HEROUT A. Vehicle re-identification for automatic video traffic surveillance[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recog-nition, Las Vegas, Jun 26-Jul 1, 2016. Washington: IEEE Computer Society, 2016: 25-31.
[16] CORMIER M, SOMMER L W, TEUTSCH M. Low resolution vehicle re-identification based on appearance features for wide area motion imagery[C]//Proceedings of the 2016 IEEE Winter Applications of Computer Vision, Lake Placid, Mar 7-10, 2016. Washington: IEEE Computer Society, 2016: 1-7.
[17] 宋克臣, 颜云辉, 陈文辉, 等. 局部二值模式方法研究与展望[J]. 自动化学报, 2013, 39(6): 730-744.
SONG K C, YAN Y H, CHEN W H, et al. Research and perspective on local binary pattern[J]. Acta Automatica Sinica, 2013, 39(6): 730-744.
[18] LI L, ZHANG X, XU Y. A network combining local fea-tures and attention mechanisms for vehicle re-identification[C]//Proceedings of the 2020 3rd International Conference on Artificial Intelligence and Pattern Recognition, Xiamen, Sep 25-27, 2020. New York: ACM, 2020: 47-50.
[19] LIU X, ZHANG S, HUANG Q, et al. RAM: a region-aware deep model for vehicle re-identification[C]//Proceedings of the 2018 IEEE International Conference on Multimedia and Expo, San Diego, Jul 23-27, 2018. Washington: IEEE Com-puter Society, 2018: 1-6.
[20] HE B, LI J, ZHAO Y, et al. Part-regularized near-duplicate vehicle re-identification[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 3997-4005.
[21] CHEN H, LAGADEC B, BREMOND F. Partition and reunion: a two-branch neural network for vehicle re-identification[C]//Proceedings of the 2019 IEEE/CVF Conference on Com-puter Vision and Pattern Recognition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 184-192.
[22] ZHU J, ZENG H, DU Y, et al. Joint feature and similarity deep learning for vehicle re-identification[J]. IEEE Access, 2018, 6: 43724-43731.
[23] WU C W, LIU C T, CHIANG C E, et al. Vehicle re-iden-tification with the space-time prior[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recog-nition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 121-128.
[24] ALFASLY S A S, HU Y, LIANG T, et al. Variational repre-sentation learning for vehicle re-identification[C]//Proceedings of the 2019 IEEE International Conference on Image Proces-sing, Taipei, China, Sep 22-25, 2019. Piscataway: IEEE, 2019: 3118-3122.
[25] HUANG P, HUANG R, HUANG J, et al. Deep feature fusion with multiple granularity for vehicle re-identification[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 80-88.
[26] WANG H, PENG J, CHEN D, et al. Attribute-guided feature learning network for vehicle reidentification[J]. IEEE Multi Media, 2020, 27(4): 112-121.
[27] WEI X S, ZHANG C L, LIU L, et al. Coarse-to-fine: a RNN-based hierarchical attention model for vehicle re-identifica-tion[C]//LNCS 11362: Proceedings of the 14th Asian Con-ference on Computer Vision, Perth, Dec 2-6, 2018. Cham:Springer, 2018: 575-591.
[28] RONG L, XU Y, ZHOU X, et al. A vehicle re-identification framework based on the improved multi-branch feature fusion network[J]. Scientific Reports, 2021, 11(1): 1-12.
[29] KHORRAMSHAHI P, PERI N, KUMAR A, et al. Attention driven vehicle re-identification and unsupervised anomaly detection for traffic understanding[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recog-nition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 239-246.
[30] RAO Y, CHEN G, LU J, et al. Counterfactual attention learning for fine-grained visual categorization and re-identi-fication[C]//Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision, Montreal, Oct 10-17, 2021. Piscataway: IEEE, 2021: 1025-1034.
[31] PAN X, LIU X, SONG B, et al. Vehicle re-identification approach combining multiple attention mechanisms and style transfer[C]//Proceedings of the 2022 3rd International Con-ference on Pattern Recognition and Machine Learning, Chengdu, Jul 22-24, 2022. Piscataway: IEEE, 2022: 65-71.
[32] LIU H, TIAN Y, YANG Y, et al. Deep relative distance lear-ning: tell the difference between similar vehicles[C]//Procee-dings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 26-Jul 1, 2016. Washington: IEEE Computer Society, 2016: 2167-2175.
[33] ZHANG Y, LIU D, ZHA Z J. Improving triplet-wise trai-ning of convolutional neural network for vehicle re-identifi-cation[C]//Proceedings of the 2017 IEEE International Con-ference on Multimedia and Expo, Hong Kong, China, Jul 10-14, 2017. Washington: IEEE Computer Society, 2017: 1386-1391.
[34] ANTONIO MARIN-REYES P, PALAZZI A, BERGA-MINI L, et al. Unsupervised vehicle re-identification using triplet networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 166-171.
[35] BAI Y, LOU Y, GAO F, et al. Group-sensitive triplet embed-ding for vehicle reidentification[J]. IEEE Transactions on Multimedia, 2018, 20(9): 2385-2399.
[36] HOU J, ZENG H, CAI L, et al. Multi-label learning with multi-label smoothing regularization for vehicle re-identifi-cation[J]. Neurocomputing, 2019, 345: 15-22.
[37] LIN W, LI Y, YANG X, et al. Multi-view learning for vehi-cle re-identification[C]//Proceedings of the 2019 IEEE Inter-national Conference on Multimedia and Expo, Shanghai, Jul 7-12, 2019. Piscataway: IEEE, 2019: 832-837.
[38] LIU X C, LIU W, MEI T, et al. A deep learning-based approach to progressive vehicle re-identification for urban surveillance[C]//LNCS 9906: Proceedings of the 14th Euro-pean Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 869-884.
[39] YAN K, TIAN Y, WANG Y, et al. Exploiting multi-grain ranking constraints for precisely searching visually-similar vehicles[C]//Proceedings of the 2017 IEEE International Con-ference on Computer Vision, Venice, Oct 22-29, 2017. Was-hington: IEEE Computer Society, 2017: 562-570.
[40] GUO H, ZHAO C, LIU Z, et al. Learning coarse-to-fine structured feature embedding for vehicle re-identification[C]//Proceedings of the 32nd AAAI Conference on Artifi-cial Intelligence, New Orleans, Feb 2-7, 2018. Palo Alto: AAAI, 2018: 6853-6860.
[41] KANACI A, ZHU X T, GONG S G. Vehicle re-identifica-tion in context[C]//LNCS 11269: Proceedings of the 40th German Conference on Pattern Recognition, Stuttgart, Oct 9-12, 2018. Cham: Springer, 2018: 377-390.
[42] LOU Y H, BAI Y, LIU J, et al.VERI-Wild: a large dataset and a new method for vehicle re-identification in the wild[C]//Proceedings of the 2019 IEEE/CVF Conference on Com-puter Vision and Pattern Recognition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 3235-3243.
[43] TANG Z, NAPHADE M, LIU M Y, et al. Cityflow: a city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification[C]//Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 15-21, 2019. Piscataway: IEEE, 2019: 8797-8806.
[44] YAO Y, ZHENG L, YANG X D, et al. Simulating content consistent vehicle datasets with attribute descent[C]//LNCS 12351: Proceedings of the 16th European Conference on Com-puter Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 775-791.
[45] 刘凯, 李浥东, 林伟鹏. 车辆再识别技术综述[J]. 智能科学与技术学报, 2020, 2(1): 10-25.
LIU K, LI Y D, LIN W P. A survey on vehicle re-identifi-cation[J]. Chinese Journal of Intelligent Science and Tech-nology, 2020, 2(1): 10-25.
[46] WANG J Y, ZHU X T, GONG S G, et al. Transferable joint attribute-identity deep learning for unsupervised person re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 2275-2284.
[47] DING Y, FAN H, XU M, et al. Adaptive exploration for unsupervised person re-identification[J]. ACM Transactions on Multimedia Computing, Communications, and Applica-tions, 2020, 16(1): 1-19.
[48] PENG J, WANG H, XU F, et al. Cross domain knowledge learning with dual-branch adversarial network for vehicle re-identification[J]. Neurocomputing, 2020, 401: 133-144.
[49] PENG J, WANG Y, WANG H, et al. Unsupervised vehicle re-identification with progressive adaptation[J]. arXiv:2006.11486, 2020.
[50] WANG Y, PENG J, WANG H, et al. Progressive learning with multi-scale attention network for cross-domain vehicle re-identification[J]. Science China Information Sciences, 2022, 65(6): 1-15.
[51] ZHOU Z, LI Y, LI J, et al. GAN-Siamese network for cross-domain vehicle re-identification in intelligent transport sys-tems[J]. IEEE Transactions on Network Science and Engi-neering, 2022. DOI: 10.1109/TNSE.2022.3199919.
[52] ZHOU Y, SHAO L. Cross-view GAN based vehicle genera-tion for re-identification[C]//Proceedings of the 28th British Machine Vision Conference, London, Sep 4-7, 2017. Durham: BMVA Press, 2017: 1-12.
[53] ZHOU Y, SHAO L. Aware attentive multi-view inference for vehicle re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 6489-6498.
[54] ZHANG F, MA Y, YUAN G, et al. Multiview image genera-tion for vehicle reidentification[J]. Applied Intelligence, 2021, 51(8): 5665-5682.
[55] WANG Q, MIN W, HAN Q, et al. Viewpoint adaptation learning with cross-view distance metric for robust vehicle re-identification[J]. Information Sciences, 2021, 564: 71-84.
[56] WU F, YAN S, SMITH J S, et al. Vehicle re-identification in still images: application of semi-supervised learning and re-ranking[J]. Signal Processing: Image Communication, 2019, 76: 261-271.
[57] LOU Y, BAI Y, LIU J, et al. Embedding adversarial lear-ning for vehicle re-identification[J]. IEEE Transactions on Image Processing, 2019, 28(8): 3794-3807.
[58] ZHU R, FANG J, XU H, et al. DCDLearn: multi-order deep cross-distance learning for vehicle re-identification[J]. arXiv:2003.11315, 2020.
[59] WANG Q, MIN W, HAN Q, et al. Inter-domain adaptation label for data augmentation in vehicle re-identification[J]. IEEE Transactions on Multimedia, 2021, 24: 1031-1041.
[60] BASHIR R M S, SHAHZAD M, FRAZ M M. VR-PROUD: vehicle re-identification using progressive unsupervised deep architecture[J]. Pattern Recognition, 2019, 90: 52-65.
[61] SONG L, WANG C, ZHANG L, et al. Unsupervised domain adaptive re-identification: theory and practice[J]. Pattern Recognition, 2020, 102: 107173.
[62] WANG H, PENG J, JIANG G, et al. Learning multiple sema-ntic knowledge for cross-domain unsupervised vehicle re-identification[C]//Proceedings of the 2021 IEEE International Conference on Multimedia and Expo, Shenzhen, Jul 5-9, 2021. Piscataway: IEEE, 2021: 1-6.
[63] DUBOURVIEUX F, LOESCH A, AUDIGIER R, et al. Im-proving unsupervised domain adaptive re-identification via source-guided selection of pseudo-labeling hyperparameters[J]. IEEE Access, 2021, 9: 149780-149795.
[64] ZHANG X, GE Y, QIAO Y, et al. Refining pseudo labels with clustering consensus over generations for unsupervised object re-identification[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 19-25, 2021. Piscataway: IEEE, 2021: 3436-3445.
[65] WANG P, DING C, TAN W, et al. Uncertainty-aware clus-tering for unsupervised domain adaptive object re-identific-ation[J]. arXiv:2108.09682, 2021.
[66] ZHENG A, SUN X, LI C, et al. Viewpoint-aware progres-sive clustering for unsupervised vehicle re-identification[J]. IEEE Transactions on Intelligent Transportation Systems, 2021, 23(8): 11422-11435.
[67] ZHU W, PENG B. Manifold-based aggregation clustering for unsupervised vehicle re-identification[J]. Knowledge-Based Systems, 2022, 235: 107624.
[68] WANG Y, WEI Y, MA R, et al. Unsupervised vehicle re-identification based on mixed sample contrastive learning[J]. Signal, Image and Video Processing, 2022,16(8): 2083-2091.
[69] ZHANG H, KOH J Y, BALDRIDGE J, et al. Cross-modal contrastive learning for text-to-image generation[C]//Procee-dings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 19-25, 2021. Piscataway: IEEE, 2021: 833-842.
[70] 安晓东, 李亚丽, 王芳. 汽车驾驶辅助系统红外与可见光融合算法综述[J]. 计算机工程与应用, 2022, 58(19): 64-75.
AN X D, LI Y Y, WANG F. Overview of infrared and visible image fusion algorithms for automotive driving assistance system[J]. Computer Engineering and Applications, 2022, 58(19): 64-75.
[71] GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial nets[C]//Proceedings of the 27th Inter-national Conference on Neural Information Processing Systems, Montreal, Dec 8-13, 2014: 2672-2680.
[72] RADFORD A, METZ L, CHINTALA S. Unsupervised rep-resentation learning with deep convolutional generative adver-sarial networks[J]. arXiv:1511.06434, 2015.
[73] CHEN X, DUAN Y, HOUTHOOFT R, et al. InfoGAN: inter-pretable representation learning by information maximizing generative adversarial nets[C]//Proceedings of the 29th Inter-national Conference on Neural Information Processing Sys-tems, Barcelona, Dec 5-10, 2016: 2172-2180.
[74] ODENA A, OLAH C, SHLENS J. Conditional image syn-thesis with auxiliary classifier GANs[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, Aug 6-11, 2017: 2642-2651.
[75] MIRZA M, OSINDERO S. Conditional generative adversarial nets[J]. arXiv:1411.1784, 2014.
[76] ALMAHAIRI A, RAJESHWAR S, SORDONI A, et al. Augmented CycleGAN: learning many-to-many mappings from unpaired data[C]//Proceedings of the 35th International Conference on Machine Learning, Stockholm, Jul 10-15, 2018: 195-204.
[77] YI S, LI J, YUAN X. DFPGAN: dual fusion path genera-tive adversarial network for infrared and visible image fusion[J]. Infrared Physics & Technology, 2021, 119: 103947.
[78] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2242-2251.
[79] YI Z L, ZHANG H, TAN P, et al. DualGAN: unsupervised dual learning for image-to-image translation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2849-2857.
[80] WEI L H, ZHANG S L, GAO W, et al. Person transfer GAN to bridge domain gap for person re-identification[C]//Procee-dings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Was-hington: IEEE Computer Society, 2018: 79-88.
[81] DENG W J, ZHENG L, YE Q X, et al. Image-image domain adaptation with preserved self-similarity and domain-dissimilarity for person re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 994-1003.
[82] KINGMA D P, WELLING M. Auto-encoding variational Bayes[J]. arXiv:1312.6114, 2013.
[83] CAO H, TAN C, GAO Z, et al. A survey on generative dif-fusion model[J]. arXiv:2209.02646, 2022.
[84] ESTER M, KRIEGEL H P, SANDER J, et al. A density-based algorithm for discovering clusters in large spatial data-bases with noise[C]//Proceedings of the 2nd International Conference on Knowledge Discovery and Data Mining, Port-land, Aug 4-8, 1996. Menlo Park: AAAI, 1996: 226-231.