Person Re-identification Based on Multi-level Feature Fusion with Overlapping Stripes

doi:10.3778/j.issn.1673-9418.2005053

Abstract

Abstract:

Most of the local features based person re-identification (Person-ReID) methods have the problem of lack of robustness and discriminability due to the distortion of information in the procedure of extraction of pedestrian features. In this paper, a novel Person-ReID algorithm based on multi-level feature fusion with overlapping stripes is proposed. In the training process, the output feature maps from different layers of the backbone network are segmented equally in the vertical axis. The features with overlapping stripes are extracted to compensate the loss of information. Three different loss functions are used for different feature vectors in the procedure of the training to minimize the intra-calss distance. Group normalization modules are applied to reducing the optimization differences within various loss functions for obtaining appropriate shared features. In the inference stage, multiple feature vectors are fused into a new feature vector, and the similarity is calculated. This algorithm is performed on Market-1501 and DukeMTMC-reID datasets with the analysis of experimental results. The proposed algorithm can improve the accuracy of Person-ReID, and the features extracted by the model have strong robustness and discriminability.

Key words: feature with overlapping stripes, feature fusion, joint learning, person re-identification (Person-ReID), deep learning

摘要：

针对行人重识别（Person-ReID）过程中，基于局部特征的方法在提取行人特征时因信息缺失导致鲁棒性和判别力不足的问题，提出一种多层级重叠条纹特征融合的行人重识别算法。训练阶段，对骨架网络不同阶段的输出特征图进行水平均等分割，再提取重叠条纹特征以弥补丢失的信息；使用三种损失函数对不同的特征向量进行监督训练，以约束类内距离。此外，设计组归一化模块来消除不同损失函数在优化方向上存在的差异，从而提取到更恰当的共享特征。推理阶段，将多个特征向量融合成一个新的特征向量，再进行相似性判断。将该方法在Market-1501、DukeMTMC-reID数据集上进行有效性实验验证并对结果进行分析。所提算法能够提高行人重识别的准确率，模型所提取的特征具有较强的鲁棒性和判别力。

关键词: 重叠条纹特征, 特征融合, 联合学习, 行人重识别（Person-ReID）, 深度学习

CHEN Fan, PENG Li. Person Re-identification Based on Multi-level Feature Fusion with Overlapping Stripes[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(9): 1753-1761.

陈璠, 彭力. 多层级重叠条纹特征融合的行人重识别[J]. 计算机科学与探索, 2021, 15(9): 1753-1761.

References

[1] LUO H, JIANG W, FAN X, et al. A survey on deep learning based person re-identification[J]. Acta Automatica Sinica, 2019, 45(11): 2032-2049.
罗浩, 姜伟, 范星, 等. 基于深度学习的行人重识别研究进展[J]. 自动化学报, 2019, 45(11): 2032-2049.
[2] ZAJDEL W, ZIVKOVIC Z, KR?SE B J A. Keeping track of humans: have I seen this person before?[C]//Proceedings of the 2005 IEEE International Conference on Robotics and Automation, Barcelona, Apr 18-22, 2005. Piscataway: IEEE, 2005: 2081-2086.
[3] LU P, DONG H S, ZHONG S, et al. Person re-identification by cross-view discriminative dictionary learning with metric embedding[J]. Journal of Computer Research and Development, 2019, 56(11): 2424-2437.
陆萍, 董虎胜, 钟珊, 等. 基于跨视角判别词典嵌入的行人再识别[J]. 计算机研究与发展, 2019, 56(11): 2424-2437.
[4] WANG Y, WANG L Q, YOU Y R, et al. Resource aware person re-identification across multiple resolutions[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 8042-8051.
[5] YU W, YANG K Y, YAO H X, et al. Exploiting the complementary strengths of multi-layer CNN features for image retrieval[J]. Neurocomputing, 2017, 237: 235-241.
[6] CHEN B, ZHA Y F, LI Y Q, et al. Person re-identification based on convolutional neural network discriminative feature learning[J]. Acta Optica Sinica, 2018, 38(7): 255-261.
陈兵, 查宇飞, 李运强, 等. 基于卷积神经网络判别特征学习的行人重识别[J]. 光学学报, 2018, 38(7): 255-261.
[7] SUN Y F, ZHENG L, YANG Y, et al. Beyond part models: person retrieval with refined part pooling (and a strong convolutional baseline)[C]//LNCS 11208: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 501-518.
[8] FU Y, WEI Y C, ZHOU Y Q, et al. Horizontal pyramid matching for person re-identification[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 8295-8302.
[9] HE K M, ZHANG X Y, REN S Q, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
[10] ZHENG F, DENG C, SUN X, et al. Pyramidal person re-identification via multi-loss dynamic training[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 8514-8522.
[11] HERMANS A, BEYER L, LEIBE B, et al. In defense of the triplet loss for person re-identification[J]. arXiv:1703.07737, 2017.
[12] WEN Y D, ZHANG K P, LI Z F, et al. A discriminative feature learning approach for deep face recognition[C]// LNCS 9911: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 499-515.
[13] LECUN Y, LEON B, ORR G B, et al. Neural networks: tricks of the trade[M]. Berlin, Heidelberg: Springer, 1996.
[14] ZHANG J. Data sphering: some properties and applications[J]. Annals of the Institute of Statistical Mathematics, 1998, 50(2): 223-240.
[15] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift [C]//Proceedings of the 32nd International Conference on Machine Learning, Lille, Jul 6-11, 2015: 448-456.
[16] ROSS S, MINEIRO P, LANGFORD J, et al. Normalized online learning[C]//Proceedings of the 29th Conference on Uncertainty in Artificial Intelligence, Bellevue, Aug 11-15, 2013: 537-545.
[17] BJORCK N, GOMES C P, SELMAN B, et al. Understanding batch normalization[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems, Montréal, Dec 3-8, 2018. Red Hook: Curran Associates, 2018: 7705-7716.
[18] BA J L, KIROS J R, HINTON G E. Layer normalization[J]. arXiv:1607.06450, 2016.
[19] ULYANOV D, VEDALDI A, LEMPITSKY V. Instance normalization: the missing ingredient for fast stylization[J]. arXiv:1607.08022, 2016.
[20] WU Y X, HE K M. Group normalization[J]. International Journal of Computer Vision, 2020, 128(3): 742-755.
[21] LI B Y, WU F, WEINBERGER K Q, et al. Positional normalization[C]//Proceedings of the 32nd Annual Conference on Neural Information Processing Systems, Vancouver, Dec 8-14, 2019. Red Hook: Curran Associates, 2019: 1620-1632.
[22] ZHENG L, SHEN L Y, TIAN L, et al. Scalable person re-identification: a benchmark[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1116-1124.
[23] RISTANI E, SOLERA F, ZOU R S, et al. Performance measures and a data set for multi-target, multi-camera tracking[C]//LNCS 9914: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 8-10 and 15-16, 2016. Cham: Springer, 2016: 17-35.
[24] XU L Z, PENG L, ZHU F Z. Pedestrian re-identification method based on multi-task pyramid overlapping matching [J]. Computer Engineering, 2021, 47(1): 239-245.
徐龙壮, 彭力, 朱凤增. 多任务金字塔重叠匹配的行人重识别方法[J]. 计算机工程, 2021, 47(1): 239-245.
[25] LI W, ZHU X T, GONG S G, et al. Harmonious attention network for person re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 2285-2294.
[26] WEI L H, ZHANG S L, YAO H T, et al. GLAD: global- local-alignment descriptor for pedestrian retrieval[C]//Procee-dings of the 25th ACM International Conference on Multimedia, Mountain View, Oct 23-27, 2017. New York: ACM, 2017: 420-428.
[27] SARFRAZ M S, SCHUMANN A, EBERLE A, et al. A pose-sensitive embedding for person re-identification with expanded cross neighborhood re-ranking[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 420-429.
[28] ERGYS R, CARLO T. Features for multi-target multi-camera tracking and re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 6036-6046.