Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion

doi:10.3778/j.issn.1673-9418.2103082

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (11): 2609-2618.DOI: 10.3778/j.issn.1673-9418.2103082

• Graphics and Image • Previous Articles Next Articles

Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion

CHEN Fan, PENG Li⁺()

Engineering Research Center of Internet of Things Technology Applications of the Ministry of Education, School of Internet of Things Engineering, Jiangnan University, Wuxi, Jiangsu 214122, China

Received:2021-03-24 Revised:2021-06-09 Online:2022-11-01 Published:2021-06-17
About author:CHEN Fan, born in 1996, M.S. candidate. Her research interests include deep learning and per-son re-identification.
PENG Li, born in 1967, Ph.D., professor, Ph.D. supervisor, member of CAAI and CCF. His re-search interests include visual Internet of things, action recognition and deep learning.
Supported by:
National Key Research and Development Program of China(2018YFD0400902);National Natural Science Foundation of China(61873112);Research Fund of Ministry of Education-China Mobile(MCM20170204);Project of Jiangsu Key Construction Laboratory of IoT Application Technology(190449);Project of Jiangsu Key Construction Laboratory of IoT Application Technology(190450)

异构分支关联特征融合的行人重识别

陈璠, 彭力⁺()

物联网技术应用教育部工程研究中心（江南大学物联网工程学院），江苏无锡 214122

通讯作者: + E-mail: penglimail2002@163.com
作者简介:陈璠（1996—），女，辽宁兴城人，硕士研究生，主要研究方向为深度学习、行人重识别。
彭力（1967—），男，河北唐山人，博士，教授，博士生导师，CAAI会员，CCF会员，主要研究方向为视觉物联网、行为识别、深度学习。
基金资助:
国家重点研发计划(2018YFD0400902);国家自然科学基金(61873112);教育部-中国移动科研基金(MCM20170204);江苏省物联网应用技术重点实验室项目(190449);江苏省物联网应用技术重点实验室项目(190450)

Abstract

Abstract:

Most of the multi-branch network based person re-identification (Person Re-ID) methods face the pro-blem of lack of heterogeneous features in the procedure of extraction of pedestrian features. In this paper, a novel Person Re-ID algorithm based on heterogeneous branch correlative features fusion is proposed. In the training stage, the attention-based OSNet is designed as the backbone sharing network, which can extract more significant and distinguished key features. The pedestrian features from branch network are segmented equally in the vertical axis. The relevant stripe features are extracted to utilize the synthesis information between different stripes. The heterogeneous features extraction module is designed to increase the structural diversity of the model for learning difference features. In the inference stage, multiple feature vectors are fused into a new feature vector, and the similarity judgment is performed. The effectiveness of the proposed algorithm is verified by experiments on Market-1501 and DukeMTMC-reID datasets, and the experiment results are analyzed. The proposed algorithm can improve the accuracy of Person Re-ID, and the features extracted by the model have strong robustness and discriminability.

Key words: stripe feature correlation, heterogeneous branch, feature fusion, person re-identification, deep learning

摘要：

针对行人重识别（Person Re-ID）过程中，多分支结构的网络模型在提取行人特征时缺乏异构特征的问题，提出一种异构分支关联特征融合的行人重识别算法。训练阶段，将OSNet与注意力机制相结合作为主干共享网络，以学习到具有更强显著性和区分性的行人关键特征；将分支网络输出的行人特征进行水平均等分割，再提取关联条纹特征，从而全面利用位于条纹间的综合信息；设计异构特征提取模块，以增加模型学习差异特征所需的结构多样性。推理阶段，将多个特征向量融合成一个新的特征向量，再进行相似性判断。将该方法在Market-1501、DukeMTMC-reID数据集上进行有效性实验验证并对结果进行分析，所提算法能够提高行人重识别的准确率，模型所提取的特征具有较强的鲁棒性和判别力。

关键词: 条纹特征关联, 异构分支, 特征融合, 行人重识别, 深度学习

CLC Number:

TP391.4

CHEN Fan, PENG Li. Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2609-2618.

陈璠, 彭力. 异构分支关联特征融合的行人重识别[J]. 计算机科学与探索, 2022, 16(11): 2609-2618.

Figures/Tables 15

References 29

[1]	姚足, 龚勋, 陈锐, 等. 面向行人重识别的局部特征研究进展、挑战与展望[J]. 自动化学报, 2021, 47(12): 2742-2760.
	YAO Z, GONG X, CHEN R, et al. Research progress, chal-lenge and prospect of local features for person re-identi- fication[J]. Acta Automatica Sinica, 2021, 47(12): 2742-2760.
[2]	SUN Y, ZHENG L, YANG Y, et al. Beyond part models: per-son retrieval with refined part pooling (and a strong con-volutional baseline)[C]// LNCS 11208: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 501-518.
[3]	WANG G S, YUAN Y F, CHEN X, et al. Learning discri-minative features with multiple granularities for person re-identification[C]// Proceedings of the ACM International Con-ference on Multimedia, Seoul, Oct 22-26, 2018. New York: ACM, 2018: 274-282.
[4]	ZHENG F, DENG C, SUN X, et al. Pyramidal person re-identification via multi-loss dynamic training[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pat-tern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 8514-8522.
[5]	SUH Y, WANG J, TANG S, et al. Part-aligned bilinear re-presentations for person re-identification[C]// LNCS 11218:Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 418-437.
[6]	ZHAO H Y, TIAN M Q, SUN S Y, et al. Spindle Net: per-son re-identification with human body region guided fea-ture decomposition and fusion[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recog-nition, Honolulu, Jul 21-26, 2017. Washington: IEEE Com-puter Society, 2017: 907-915.
[7]	ZHAO L M, LI X, ZHUANG Y T, et al. Deeply-learned part-aligned representations for person re-identification[C]// Pro-ceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 3239-3248.
[8]	ZHENG L, HUANG Y J, LU H C, et al. Pose invariant em-bedding for deep person re-identification[J]. IEEE Transac-tions on Image Processing, 2017, 28(9): 4500-4509.
[9]	贲晛烨, 徐森, 王科俊. 行人步态的特征表达及识别综述[J]. 模式识别与人工智能, 2012, 25(1): 71-81.
	BEN X Y, XU S, WANG K J. Review on pedestrian gait feature expression and recognition[J]. Pattern Recognition and Artificial Intelligence, 2012, 25(1): 71-81.
[10]	YAO H, ZHANG S, ZHANG Y, et al. Deep representation learning with part loss for person re-identification[J]. IEEE Transactions on Image Processing, 2019, 28(6): 2860-2871. DOI URL
[11]	ZHOU K, YANG Y, CAVALLARO A, et al. Omni-scale feature learning for person re-identification[C]// Proceedings of the 2019 IEEE/CVF International Conference on Com-puter Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 3701-3711.
[12]	SONG C F, HUANG Y, OUYANG W L, et al. Mask-guided contrastive attention model for person re-identification[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 1179-1188.
[13]	CHEN T L, DING S J, XIE J Y, et al. ABD-Net: attentive but diverse person re-identification[C]// Proceedings of the 2019 IEEE International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 8350-8360.
[14]	李文涛, 彭力. 多尺度通道注意力融合网络的小目标检测算法[J]. 计算机科学与探索, 2021, 15(12): 2390-2400.
	LI W T, PENG L. Small objects detection algorithm with multi-scale channel attention fusion network[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(12): 2390-2400.
[15]	卓天天, 桑庆兵. 注意力机制与复合卷积在手写识别中的应用[J]. 计算机科学与探索, 2022, 16(4): 888-897. DOI
	ZHUO T T, SANG Q B. Application of attention mecha-nism and composite convolution in handwriting recognition[J]. Journal of Frontiers of Computer Science and Techno-logy, 2022, 16(4): 888-897.
[16]	HU J, SHEN L, SUN G, et al. Squeeze-and-excitation net-works[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 7132-7141.
[17]	HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]// Proceedings of the 2016 IEEE Con-ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer So-ciety, 2016: 770-778.
[18]	WEN Y D, ZHANG K P, LI Z F, et al. A discriminative fea-ture learning approach for deep face recognition[C]// LNCS 9911: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Sprin-ger, 2016: 499-515.
[19]	CHENG D, GONG Y H, ZHOU S P, et al. Person re-identification by multi-channel parts-based CNN with im-proved triplet loss function[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion, Las Vegas, Jun 27-30, 2016. Washington: IEEE Compu-ter Society, 2016: 1335-1344.
[20]	ZHENG L, SHEN L, TIAN L, et al. Scalable person re-identification: a benchmark[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1116-1124.
[21]	RISTANI E, SOLERA F, ZOU R S, et al. Performance measures and a data set for multi-target, multi-camera trac-king[C]// LNCS 9914: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 8-10, 15-16, 2016. Cham: Springer, 2016: 17-35.
[22]	XIONG R, YANG Y, HE D, et al. On layer normalization in the transformer architecture[C]// Proceedings of the 37th In-ternational Conference on Machine Learning, Jul 13-18, 2020: 10524-10533.
[23]	XIA N, GONG Y, ZHANG Y Z, et al. Second-order non-local attention networks for person re-identification[C]// Procee-dings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 3759-3768.
[24]	QUAN R, DONG X, WU Y, et al. Auto-ReID: searching for a part-aware ConvNet for person re-identification[C]// Pro-ceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscata-way: IEEE, 2019: 3749-3758.
[25]	HE T, ZHANG Z, ZHANG H, et al. Bag of tricks for image classification with convolutional neural networks[C]// Pro-ceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Pisca-taway: IEEE, 2019: 558-567.
[26]	WANG G, LAI J, HUANG P, et al. Spatial-temporal person re-identification[C]// Proceedings of the 33rd AAAI Confer-ence on Artificial Intelligence, the 31st Innovative Applica-tions of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelli-gence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 8933-8940.
[27]	CHEN B, DENG W, HU J. Mixed high-order attention net-work for person re-identification[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Se-oul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 371-381.
[28]	YANG W, HUANG H, ZHANG Z, et al. Towards rich feature discovery with class activation maps augmentation for person re-identification[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1389-1398.
[29]	HOU R, MA B, CHANG H, et al. Interaction and aggrega-tion network for person re-identification[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 9317-9326.

模型	rank-1	mAP
v1	93.4	85.9
v1+v2	94.4	87.7
v1+v3	95.6	88.9
v1+v2+v3	96.7	89.5

模型	rank-1	mAP
v1	93.4	85.9
v1+v2	94.4	87.7
v1+v3	95.6	88.9
v1+v2+v3	96.7	89.5

模型	rank-1	mAP
v1	88.5	76.9
v1+v2	89.8	78.7
v1+v3	89.9	79.5
v1+v2+v3	91.7	81.6

模型	rank-1	mAP
v1	88.5	76.9
v1+v2	89.8	78.7
v1+v3	89.9	79.5
v1+v2+v3	91.7	81.6

注意力机制	rank-1	mAP
无	96.1	89.0
有	96.7	89.5

Person Re-identification Based on Heterogeneous Branch Correlative Features Fusion

异构分支关联特征融合的行人重识别

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 15

References 29

Related Articles 15

Recommended Articles

Metrics

模型	rank-1	mAP
SONA	95.7	88.7
Auto-ReID	94.5	85.1
CAMA	94.7	84.5
MGN	95.7	86.9
Bag Of Tricks	94.5	85.9
St-ReID	98.1	87.6
MHN	95.1	85.9
OSNet	94.8	84.9
IAN	94.4	83.1
Ours	96.7	89.5

模型	rank-1	mAP
SONA	89.3	78.1
Auto-ReID	88.5	75.1
CAMA	85.8	72.9
MGN	88.7	78.4
Bag Of Tricks	86.4	76.4
St-ReID	94.4	83.9
MHN	89.1	77.2
OSNet	88.6	73.5
IAN	87.1	73.4
Ours	91.7	81.6

[1]	LYU Xiaoqi, JI Ke, CHEN Zhenxiang, SUN Runyuan, MA Kun, WU Jun, LI Yidong. Expert Recommendation Algorithm Combining Attention and Recurrent Neural Network [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2068-2077.
[2]	ZHANG Xiangping, LIU Jianxun. Overview of Deep Learning-Based Code Representation and Its Applications [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 2011-2029.
[3]	LI Dongmei, LUO Sisi, ZHANG Xiaoping, XU Fu. Review on Named Entity Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1954-1968.
[4]	REN Ning, FU Yan, WU Yanxia, LIANG Pengju, HAN Xi. Review of Research on Imbalance Problem in Deep Learning Applied to Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1933-1953.
[5]	YANG Caidong, LI Chengyang, LI Zhongbo, XIE Yongqiang, SUN Fangwei, QI Jin. Review of Image Super-resolution Reconstruction Algorithms Based on Deep Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(9): 1990-2010.
[6]	ZENG Fanzhi, XU Luqian, ZHOU Yan, ZHOU Yuexia, LIAO Junwei. Review of Knowledge Tracing Model for Intelligent Education [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1742-1763.
[7]	AN Fengping, LI Xiaowei, CAO Xiang. Medical Image Classification Algorithm Based on Weight Initialization-Sliding Window CNN [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1885-1897.
[8]	LIU Yi, LI Mengmeng, ZHENG Qibin, QIN Wei, REN Xiaoguang. Survey on Video Object Tracking Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515.
[9]	ZHAO Xiaoming, YANG Yijiao, ZHANG Shiqing. Survey of Deep Learning Based Multimodal Emotion Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1479-1503.
[10]	XIA Hongbin, XIAO Yifei, LIU Yuan. Long Text Generation Adversarial Network Model with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610.
[11]	PENG Hao, LI Xiaoming. Multi-scale Selection Pyramid Networks for Small-Sample Target Detection Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1649-1660.
[12]	SUN Fangwei, LI Chengyang, XIE Yongqiang, LI Zhongbo, YANG Caidong, QI Jin. Review of Deep Learning Applied to Occluded Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1243-1259.
[13]	LIU Yafen, ZHENG Yifeng, JIANG Lingyi, LI Guohe, ZHANG Wenjie. Survey on Pseudo-Labeling Methods in Deep Semi-supervised Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1279-1290.
[14]	ZHAO Yunji, FAN Cunliang, ZHANG Xinliang. Object Tracking Algorithm with Fusion of Multi-feature and Channel Awareness [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1417-1428.
[15]	CHENG Weiyue, ZHANG Xueqin, LIN Kezheng, LI Ao. Deep Convolutional Neural Network Algorithm Fusing Global and Local Features [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1146-1154.