异构分支关联特征融合的行人重识别

doi:10.3778/j.issn.1673-9418.2103082

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (11): 2609-2618.DOI: 10.3778/j.issn.1673-9418.2103082

异构分支关联特征融合的行人重识别

陈璠, 彭力⁺()

物联网技术应用教育部工程研究中心（江南大学物联网工程学院），江苏无锡 214122

收稿日期:2021-03-24 修回日期:2021-06-09 出版日期:2022-11-01 发布日期:2021-06-17
通讯作者: + E-mail: penglimail2002@163.com
作者简介:陈璠（1996—），女，辽宁兴城人，硕士研究生，主要研究方向为深度学习、行人重识别。
彭力（1967—），男，河北唐山人，博士，教授，博士生导师，CAAI会员，CCF会员，主要研究方向为视觉物联网、行为识别、深度学习。
基金资助:
国家重点研发计划(2018YFD0400902);国家自然科学基金(61873112);教育部-中国移动科研基金(MCM20170204);江苏省物联网应用技术重点实验室项目(190449);江苏省物联网应用技术重点实验室项目(190450)

Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion

CHEN Fan, PENG Li⁺()

Engineering Research Center of Internet of Things Technology Applications of the Ministry of Education, School of Internet of Things Engineering, Jiangnan University, Wuxi, Jiangsu 214122, China

Received:2021-03-24 Revised:2021-06-09 Online:2022-11-01 Published:2021-06-17
About author:CHEN Fan, born in 1996, M.S. candidate. Her research interests include deep learning and per-son re-identification.
PENG Li, born in 1967, Ph.D., professor, Ph.D. supervisor, member of CAAI and CCF. His re-search interests include visual Internet of things, action recognition and deep learning.
Supported by:
National Key Research and Development Program of China(2018YFD0400902);National Natural Science Foundation of China(61873112);Research Fund of Ministry of Education-China Mobile(MCM20170204);Project of Jiangsu Key Construction Laboratory of IoT Application Technology(190449);Project of Jiangsu Key Construction Laboratory of IoT Application Technology(190450)

摘要/Abstract

摘要：

针对行人重识别（Person Re-ID）过程中，多分支结构的网络模型在提取行人特征时缺乏异构特征的问题，提出一种异构分支关联特征融合的行人重识别算法。训练阶段，将OSNet与注意力机制相结合作为主干共享网络，以学习到具有更强显著性和区分性的行人关键特征；将分支网络输出的行人特征进行水平均等分割，再提取关联条纹特征，从而全面利用位于条纹间的综合信息；设计异构特征提取模块，以增加模型学习差异特征所需的结构多样性。推理阶段，将多个特征向量融合成一个新的特征向量，再进行相似性判断。将该方法在Market-1501、DukeMTMC-reID数据集上进行有效性实验验证并对结果进行分析，所提算法能够提高行人重识别的准确率，模型所提取的特征具有较强的鲁棒性和判别力。

关键词: 条纹特征关联, 异构分支, 特征融合, 行人重识别, 深度学习

Abstract:

Most of the multi-branch network based person re-identification (Person Re-ID) methods face the pro-blem of lack of heterogeneous features in the procedure of extraction of pedestrian features. In this paper, a novel Person Re-ID algorithm based on heterogeneous branch correlative features fusion is proposed. In the training stage, the attention-based OSNet is designed as the backbone sharing network, which can extract more significant and distinguished key features. The pedestrian features from branch network are segmented equally in the vertical axis. The relevant stripe features are extracted to utilize the synthesis information between different stripes. The heterogeneous features extraction module is designed to increase the structural diversity of the model for learning difference features. In the inference stage, multiple feature vectors are fused into a new feature vector, and the similarity judgment is performed. The effectiveness of the proposed algorithm is verified by experiments on Market-1501 and DukeMTMC-reID datasets, and the experiment results are analyzed. The proposed algorithm can improve the accuracy of Person Re-ID, and the features extracted by the model have strong robustness and discriminability.

Key words: stripe feature correlation, heterogeneous branch, feature fusion, person re-identification, deep learning

中图分类号:

TP391.4

陈璠, 彭力. 异构分支关联特征融合的行人重识别[J]. 计算机科学与探索, 2022, 16(11): 2609-2618.

CHEN Fan, PENG Li. Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2609-2618.

图/表 15

图1 本文模型结构框图

Fig.1 Block diagram of model structure

图2 SAM模块

Fig.2 Structure of SAM

图3 CAM模块

Fig.3 Structure of CAM

图4 条纹特征关联模块

Fig.4 Stripe feature correlation module

图5 异构特征提取模块

Fig.5 Structure of heterogeneous feature extraction module

图6 单一ID损失与多重ID损失

Fig.6 Single ID loss vs multiple ID loss

表1 Experimental results with different branches on Market-1501 dataset 单位：%

Table 1

模型	rank-1	mAP
v1	93.4	85.9
v1+v2	94.4	87.7
v1+v3	95.6	88.9
v1+v2+v3	96.7	89.5

表2 Experimental results with different branches on DukeMTMC-reID dataset 单位：%

Table 2

模型	rank-1	mAP
v1	88.5	76.9
v1+v2	89.8	78.7
v1+v3	89.9	79.5
v1+v2+v3	91.7	81.6

表3 Experimental results of attention modules on Market-1501 dataset 单位：%

Table 3

注意力机制	rank-1	mAP
无	96.1	89.0
有	96.7	89.5

表4 Experimental results of attention modules on DukeMTMC-reID dataset 单位：%

Table 4

注意力机制	rank-1	mAP
无	91.0	81.3
有	91.7	81.6

表5 Experimental results with different losses on Market-1501 dataset 单位：%

Table 5

方法	rank-1	mAP
多重ID损失	94.8	85.9
单一ID损失	96.7	89.5

表6 Experimental results with different losses on DukeMTMC-reID dataset 单位：%

Table 6

方法	rank-1	mAP
多重ID损失	89.9	77.8
单一ID损失	91.7	81.6

表7 Comparison with mainstream algorithms on Market-1501 dataset 单位：%

Table 7

模型	rank-1	mAP
SONA	95.7	88.7
Auto-ReID	94.5	85.1
CAMA	94.7	84.5
MGN	95.7	86.9
Bag Of Tricks	94.5	85.9
St-ReID	98.1	87.6
MHN	95.1	85.9
OSNet	94.8	84.9
IAN	94.4	83.1
Ours	96.7	89.5

表8 Comparison with mainstream algorithms on DukeMTMC-reID dataset 单位：%

Table 8

模型	rank-1	mAP
SONA	89.3	78.1
Auto-ReID	88.5	75.1
CAMA	85.8	72.9
MGN	88.7	78.4
Bag Of Tricks	86.4	76.4
St-ReID	94.4	83.9
MHN	89.1	77.2
OSNet	88.6	73.5
IAN	87.1	73.4
Ours	91.7	81.6

图7 本文模型在Market-1501数据集上部分测试结果

Fig.7 Part of test results of proposed model on Market-1501 dataset

参考文献 29

[1]	姚足, 龚勋, 陈锐, 等. 面向行人重识别的局部特征研究进展、挑战与展望[J]. 自动化学报, 2021, 47(12): 2742-2760.
	YAO Z, GONG X, CHEN R, et al. Research progress, chal-lenge and prospect of local features for person re-identi- fication[J]. Acta Automatica Sinica, 2021, 47(12): 2742-2760.
[2]	SUN Y, ZHENG L, YANG Y, et al. Beyond part models: per-son retrieval with refined part pooling (and a strong con-volutional baseline)[C]// LNCS 11208: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 501-518.
[3]	WANG G S, YUAN Y F, CHEN X, et al. Learning discri-minative features with multiple granularities for person re-identification[C]// Proceedings of the ACM International Con-ference on Multimedia, Seoul, Oct 22-26, 2018. New York: ACM, 2018: 274-282.
[4]	ZHENG F, DENG C, SUN X, et al. Pyramidal person re-identification via multi-loss dynamic training[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pat-tern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 8514-8522.
[5]	SUH Y, WANG J, TANG S, et al. Part-aligned bilinear re-presentations for person re-identification[C]// LNCS 11218:Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 418-437.
[6]	ZHAO H Y, TIAN M Q, SUN S Y, et al. Spindle Net: per-son re-identification with human body region guided fea-ture decomposition and fusion[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recog-nition, Honolulu, Jul 21-26, 2017. Washington: IEEE Com-puter Society, 2017: 907-915.
[7]	ZHAO L M, LI X, ZHUANG Y T, et al. Deeply-learned part-aligned representations for person re-identification[C]// Pro-ceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 3239-3248.
[8]	ZHENG L, HUANG Y J, LU H C, et al. Pose invariant em-bedding for deep person re-identification[J]. IEEE Transac-tions on Image Processing, 2017, 28(9): 4500-4509.
[9]	贲晛烨, 徐森, 王科俊. 行人步态的特征表达及识别综述[J]. 模式识别与人工智能, 2012, 25(1): 71-81.
	BEN X Y, XU S, WANG K J. Review on pedestrian gait feature expression and recognition[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(1): 71-81.
[10]	YAO H, ZHANG S, ZHANG Y, et al. Deep representation learning with part loss for person re-identification[J]. IEEE Transactions on Image Processing, 2019, 28(6): 2860-2871. DOI URL
[11]	ZHOU K, YANG Y, CAVALLARO A, et al. Omni-scale feature learning for person re-identification[C]// Proceedings of the 2019 IEEE/CVF International Conference on Com-puter Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 3701-3711.
[12]	SONG C F, HUANG Y, OUYANG W L, et al. Mask-guided contrastive attention model for person re-identification[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 1179-1188.
[13]	CHEN T L, DING S J, XIE J Y, et al. ABD-Net: attentive but diverse person re-identification[C]// Proceedings of the 2019 IEEE International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 8350-8360.
[14]	李文涛, 彭力. 多尺度通道注意力融合网络的小目标检测算法[J]. 计算机科学与探索, 2021, 15(12): 2390-2400.
	LI W T, PENG L. Small objects detection algorithm with multi-scale channel attention fusion network[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(12): 2390-2400.
[15]	卓天天, 桑庆兵. 注意力机制与复合卷积在手写识别中的应用[J]. 计算机科学与探索, 2022, 16(4): 888-897. DOI
	ZHUO T T, SANG Q B. Application of attention mecha-nism and composite convolution in handwriting recognition[J]. Journal of Frontiers of Computer Science and Techno-logy, 2022, 16(4): 888-897.
[16]	HU J, SHEN L, SUN G, et al. Squeeze-and-excitation net-works[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 7132-7141.
[17]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Con-ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer So-ciety, 2016: 770-778.
[18]	WEN Y D, ZHANG K P, LI Z F, et al. A discriminative fea-ture learning approach for deep face recognition[C]// LNCS 9911: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Sprin-ger, 2016: 499-515.
[19]	CHENG D, GONG Y H, ZHOU S P, et al. Person re-identification by multi-channel parts-based CNN with im-proved triplet loss function[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion, Las Vegas, Jun 27-30, 2016. Washington: IEEE Compu-ter Society, 2016: 1335-1344.
[20]	ZHENG L, SHEN L, TIAN L, et al. Scalable person re-identification: a benchmark[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1116-1124.
[21]	RISTANI E, SOLERA F, ZOU R S, et al. Performance measures and a data set for multi-target, multi-camera trac-king[C]// LNCS 9914: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 8-10, 15-16, 2016. Cham: Springer, 2016: 17-35.
[22]	XIONG R, YANG Y, HE D, et al. On layer normalization in the transformer architecture[C]// Proceedings of the 37th In-ternational Conference on Machine Learning, Jul 13-18, 2020: 10524-10533.
[23]	XIA N, GONG Y, ZHANG Y Z, et al. Second-order non-local attention networks for person re-identification[C]// Procee-dings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 3759-3768.
[24]	QUAN R, DONG X, WU Y, et al. Auto-ReID: searching for a part-aware ConvNet for person re-identification[C]// Pro-ceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscata-way: IEEE, 2019: 3749-3758.
[25]	HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]// Pro-ceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Pisca-taway: IEEE, 2019: 558-567.
[26]	WANG G, LAI J, HUANG P, et al. Spatial-temporal person re-identification[C]// Proceedings of the 33rd AAAI Confer-ence on Artificial Intelligence, the 31st Innovative Applica-tions of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelli-gence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 8933-8940.
[27]	CHEN B, DENG W, HU J. Mixed high-order attention net-work for person re-identification[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Se-oul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 371-381.
[28]	YANG W, HUANG H, ZHANG Z, et al. Towards rich feature discovery with class activation maps augmentation for person re-identification[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1389-1398.
[29]	HOU R, MA B, CHANG H, et al. Interaction and aggrega-tion network for person re-identification[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 9317-9326.

异构分支关联特征融合的行人重识别

Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 15

参考文献 29

相关文章 15

编辑推荐

Metrics

[1]	吕晓琦, 纪科, 陈贞翔, 孙润元, 马坤, 邬俊, 李浥东. 结合注意力与循环神经网络的专家推荐算法[J]. 计算机科学与探索, 2022, 16(9): 2068-2077.
[2]	张祥平, 刘建勋. 基于深度学习的代码表征及其应用综述[J]. 计算机科学与探索, 2022, 16(9): 2011-2029.
[3]	李冬梅, 罗斯斯, 张小平, 许福. 命名实体识别方法研究综述[J]. 计算机科学与探索, 2022, 16(9): 1954-1968.
[4]	任宁, 付岩, 吴艳霞, 梁鹏举, 韩希. 深度学习应用于目标检测中失衡问题研究综述[J]. 计算机科学与探索, 2022, 16(9): 1933-1953.
[5]	杨才东, 李承阳, 李忠博, 谢永强, 孙方伟, 齐锦. 深度学习的图像超分辨率重建技术综述[J]. 计算机科学与探索, 2022, 16(9): 1990-2010.
[6]	曾凡智, 许露倩, 周燕, 周月霞, 廖俊玮. 面向智慧教育的知识追踪模型研究综述[J]. 计算机科学与探索, 2022, 16(8): 1742-1763.
[7]	安凤平, 李晓薇, 曹翔. 权重初始化-滑动窗口CNN的医学图像分类[J]. 计算机科学与探索, 2022, 16(8): 1885-1897.
[8]	刘艺, 李蒙蒙, 郑奇斌, 秦伟, 任小广. 视频目标跟踪算法综述[J]. 计算机科学与探索, 2022, 16(7): 1504-1515.
[9]	赵小明, 杨轶娇, 张石清. 面向深度学习的多模态情感识别研究进展[J]. 计算机科学与探索, 2022, 16(7): 1479-1503.
[10]	夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610.
[11]	彭豪, 李晓明. 多尺度选择金字塔网络的小样本目标检测算法[J]. 计算机科学与探索, 2022, 16(7): 1649-1660.
[12]	孙方伟, 李承阳, 谢永强, 李忠博, 杨才东, 齐锦. 深度学习应用于遮挡目标检测算法综述[J]. 计算机科学与探索, 2022, 16(6): 1243-1259.
[13]	刘雅芬, 郑艺峰, 江铃燚, 李国和, 张文杰. 深度半监督学习中伪标签方法综述[J]. 计算机科学与探索, 2022, 16(6): 1279-1290.
[14]	赵运基, 范存良, 张新良. 融合多特征和通道感知的目标跟踪算法[J]. 计算机科学与探索, 2022, 16(6): 1417-1428.
[15]	程卫月, 张雪琴, 林克正, 李骜. 融合全局与局部特征的深度卷积神经网络算法[J]. 计算机科学与探索, 2022, 16(5): 1146-1154.