Person Re-identification Combined with Clothing Information Transfer

doi:10.3778/j.issn.1673-9418.2006104

Abstract

Abstract:

Person re-identification (Re-ID) is an identification method based on the overall characteristics of human body, which is usually used to judge whether there is a specific person in the image or video sequence. The clothing characteristics of persons have very important impact on Re-ID methods. But in some special scenarios, for example, a criminal maybe change his clothes in order to escape tracking, then his clothes will be different in the query image from those in the gallery image, which will cause the existing Re-ID algorithms difficult to match the target person in the gallery image. In order to solve this problem, a Re-ID model based on clothing information transfer is proposed in this paper. Given a query image of a target person, all the persons are re-dressed in the gallery image with the clothing feature in the query image, therefore the difference of clothing feature in person matching process can be eliminated. Finally, the model is verified on the dataset PRDDC (person Re-ID with different clothes), and experimental results show the significant improvement of the proposed algorithm compared with the algorithms without clothing feature transfer module.

Key words: person re-identification (Re-ID), feature transfer, image matching

摘要：

行人重新识别（Re-ID）是一种基于人体整体特征的身份识别方法，是指在图像或者视频序列中判断是否存在特定行人的技术。对于行人重识别方法来说，行人所穿着的衣服特征具有十分重要的影响。然而在某些特殊情况下（例如罪犯为了摆脱跟踪，更换衣服后逃跑等），目标行人在查询图像和图像库中的衣服特征是不同的，这使得目前的行人重识别方法难以将目标人物的图像与图像库中的人物相匹配。为了解决这一问题，提出了一种基于衣服特征迁移的行人重识别方法。给定一个目标人物的查询图像，将该查询图像行人的衣服特征迁移到图像库中的所有行人身上，以此消除衣服特征的差异。最后，该模型在数据集PRDDC上进行了验证，实验结果表明，与没有服装特征转移模块的算法相比，该算法有显著的改进。

关键词: 行人重识别（Re-ID）, 特征迁移, 图像匹配

YUAN Chunmiao, NIU Ying, GUO Tao, LI Xin. Person Re-identification Combined with Clothing Information Transfer[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(9): 1740-1752.

苑春苗, 牛瑛, 郭涛, 李鑫. 融合衣服特征迁移的行人重识别方法[J]. 计算机科学与探索, 2021, 15(9): 1740-1752.

References

[1] ZHENG L, YANG Y, HAUPTMANN A G. Person re-identi-fication: past, present and future[J]. arXiv:1610.02984, 2016.
[2] ZHENG L, BIE Z, SUN Y F, et al. MARS: a video benchmark for large-scale person re-identification[C]//LNCS 9910: Pro-ceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 868-884.
[3] WEI L H, ZHANG S L, GAO W, et al. Person transfer GAN to bridge domain gap for person re-identification[C]//Pro-ceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 79-88.
[4] KALAYEH M M, BASARAN E, G?KMEN M, et al. Human semantic parsing for person re-identification[C]//Proceed-ings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Pisca-taway: IEEE, 2018: 1062-1071.
[5] DAI Z Z, CHEN M Q, GU X D, et al. Batch DropBlock net-work for person re-identification and beyond[C]//Proceedings of the 2019 IEEE/CVF International Conference on Com-puter Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 3690-3700.
[6] FU Y, WEI Y C, WANG G S, et al. Self-similarity grouping: a simple unsupervised cross domain adaptation approach for person re-identification[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 6111-6120.
[7] LASSNER C, PONS-MOLL G, GEHLER P V. A genera-tive model of people in clothing[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 853-862.
[8] MA L, JIA X, SUN Q, et al. Pose guided person image genera-tion[C]//Proceedings of the 31st Annual Conference on Neural Information Processing Systems, Long Beach, Dec 4-9, 2017. Red Hook: Curran Associates, 2017: 406-416.
[9] HAN X T, WU Z X, WU Z, et al. Viton: an image-based virtual try-on network[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 7543-7552.
[10] YOO D, KIM N, PARK S, et al. Pixel-level domain transfer [C]//LNCS 9912: Proceedings of the 14th European Confer-ence on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 517-532.
[11] GHEISSARI N, SEBASTIAN T B, HARTLEY R I. Person reidentification using spatiotemporal appearance[C]//Proc-eedings of the 2006 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, Jun 17-22, 2006. Washington: IEEE Computer Society, 2006: 1528-1535.
[12] ZHAO R, OUYANG W L, WANG X G. Unsupervised salience learning for person re-identification[C]//Proceedings of the 2013 IEEE Conference on Computer Vision and Pattern Reco-gnition, Portland, Jun 23-28, 2013. Piscataway: IEEE, 2013: 3586-3593.
[13] KALANTIDIS Y, KENNEDY L, LI L J. Getting the look: clothing recognition and segmentation for automatic product suggestions in everyday photos[C]//Proceedings of the 3rd ACM Conference on International Conference on Multimedia Retrieval. New York: ACM, 2013: 105-112.
[14] DENG W, ZHENG L, YE Q, et al. Image-image domain adaptation with preserved self-similarity and domain-dissi-milarity for person re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway:IEEE, 2018: 994-1003.
[15] CHEN W, CHEN X, ZHANG J, et al. Beyond triplet loss: a deep quadruplet network for person re-identification[C]//Pro-ceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Pisca-taway: IEEE, 2017: 1320-1329.
[16] ANDRILUKA M, ROTH S, SCHIELE B, et al. Pictorial structures revisited: people detection and articulated pose estimation[C]//Proceedings of the 2009 IEEE Computer Society Conference on Computer Vision and Pattern Recogni-tion, Miami, Jun 20-25, 2009. Washington: IEEE Computer Society, 2009: 1014-1021.
[17] CHEN X J, YUILLE A L. Articulated pose estimation by a graphical model with image dependent pairwise relations[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Montreal, Dec 8-13, 2014. Red Hook: Curran Associates, 2014: 1736-1744.
[18] OUYANG W L, CHU X, WANG X G. Multi-source deep learning for human pose estimation[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Reco-gnition, Columbus, Jun 23-28, 2014. Piscataway: IEEE, 2014: 2337-2344.
[19] YANG Y, RAMANAN D. Articulated pose estimation with flexible mixtures-of-parts[C]//Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, Jun 20-25, 2011. Piscataway: IEEE, 2011: 1385-1392.
[20] TOSHEV A, SZEGEDY C. DeepPose: human pose estima-tion via deep neural networks[C]//Proceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recogni-tion, Columbus, Jun 23-28, 2014. Piscataway: IEEE, 2014: 1653-1660.
[21] WEI S E, RAMAKRISHNA V, KANADE T, et al. Convo-lutional pose machines[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Piscataway: IEEE, 2016: 4724-4732.
[22] NEWELL A, YANG K, DENG J. Stacked hourglass networks for human pose estimation[C]//LNCS 9912: Proceedings of the 14th European Conference on Computer Vision, Ams-terdam, Oct 11-14, 2016. Cham: Springer, 2016: 483-499.
[23] CHEN L C, YANG Y, WANG J, et al. Attention to scale: scale-aware semantic image segmentation[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Piscataway: IEEE, 2016: 3640-3649.
[24] LIANG X D, XU C Y, SHEN X H, et al. Human parsing with contextualized convolutional neural network[C]//Proc-eedings of the 2015 IEEE International Conference on Com-puter Vision, Santiago, Dec 7-13, 2015. Piscataway: IEEE,2015: 1386-1394.
[25] LIU S, LIANG X D, LIU L Q, et al. Matching-CNN meets KNN: quasi-parametric human parsing[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Piscataway: IEEE, 2015: 1419-1427.
[26] SIMO-SERRA E, FIDLER S, MORENO-NOGUER F, et al. A high performance CRF model for clothes parsing[C]//LNCS 9005: Proceedings of the 12th Asian Conference on Computer Vision, Singapore, Nov 1-5, 2014. Cham: Springer, 2014: 64-81.
[27] XIA F T, WANG P, CHEN L C, et al. Zoom better to see clearer: human and object parsing with hierarchical auto-zoom net[C]//LNCS 9909: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 648-663.
[28] YAMAGUCHI K, KIAPOUR H M, BERG T L. Paper doll parsing: retrieving similar styles to parse clothing items[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Piscataway: IEEE, 2013: 3519-3526.
[29] YAMAGUCHI K, KIAPOUR M H, ORTIZ L E, et al. Parsing clothing in fashion photographs[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Jun 16-21, 2012. Piscataway: IEEE, 2012: 3570-3577.
[30] WANG P, SHEN X H, LIN Z L, et al. Joint object and part segmentation using deep learned potentials[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Piscataway: IEEE, 2015: 1573-1581.
[31] ZHENG S, JAYASUMANA S, ROMERA-PAREDES B, et al. Conditional random fields as recurrent neural networks[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Piscataway: IEEE, 2015: 1529-1537.
[32] ZHU S Z, FIDLER S, URTASUN R, et al. Be your own prada: fashion synthesis with structural coherence[C]//Procee-dings of the 2017 International Conference on Computer Vision, Venice, Oct 22-29, 2017. Piscataway: IEEE, 2017: 1689-1697.
[33] YANG S, AMBERT T, PAN Z, et al. Detailed garment recovery from a single-view image[J]. arXiv:1608.01250, 2016.
[34] RAJ A, SANGKLOY P, CHANG H W, et al. SwapNet: image based garment transfer[C]//LNCS 11216: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 679-695.
[35] XIAO T, LI S, WANG B C, et al. Joint detection and identi-fication feature learning for person search[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Piscataway: IEEE, 2017: 3376-3385.
[36] WU S, CHEN Y C, LI X, et al. An enhanced deep feature representation for person re-identification[C]//Proceedings of the 2016 IEEE Winter Conference on Applications of Computer Vision. Piscataway: IEEE, 2016: 1-8.
[37] REN S Q, HE K M, GIRSHICK R B, et al. Faster R-CNN: towards real-time object detection with region proposal networks[C]//Proceedings of the 28th Annual Conference on Neural Information Processing Systems, Montreal, Dec 7-12, 2015. Red Hook: Curran Associates, 2015: 91-99.
[38] SONG S, ZHANG W, LIU J, et al. Unsupervised person image generation with semantic parsing transformation[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 2357-2366.
[39] CAO Z, SIMON T, WEI S E, et al. Realtime multi-person 2D pose estimation using part affinity fields[C]//Proceed-ings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Piscataway: IEEE, 2017: 1302-1310.
[40] LIANG X D, GONG K, SHEN X H, et al. Look into person: joint body parsing & pose estimation network and a new benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 41(4): 871-885.
[41] LONG J, SHELHAMER E, DARRELL T. Fully convolu-tional networks for semantic segmentation[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Piscataway: IEEE, 2015: 3431-3440.
[42] BADRINARAYANAN V, KENDALL A, CIPOLLA R. SegNet: a deep convolutional encoder-decoder architecture for image segmentation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39(12): 2481-2495.
[43] CHEN L C, PAPANDREOU G, KOKKINOS I, et al. DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs[J]. IEEE Trans-actions on Pattern Analysis and Machine Intelligence, 2017, 40(4): 834-848.
[44] PFISTER T, CHARLES J, ZISSERMAN A. Flowing convnets for human pose estimation in videos[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Piscataway: IEEE, 2015: 1913-1921.
[45] GONG K, LIANG X D, ZHANG D Y, et al. Look into person: self-supervised structure-sensitive learning and a new ben-chmark for human parsing[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recogni-tion, Honolulu, Jul 21-26, 2017. Piscataway: IEEE, 2017: 6757-6765.
[46] FELZENSZWALB P F, GIRSHICK R B, MCALLESTER D A. Cascade object detection with deformable part models [C]//Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, Jun 13-18, 2010. Piscataway: IEEE, 2010: 2241-2248.
[47] DOLLáR P, APPEL R, BELONGIE S, et al. Fast feature pyramids for object detection[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(8): 1532-1545.
[48] NAM W, DOLLáR P, HAN J H. Local decorrelation for improved pedestrian detection[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Montreal, Dec 8-13, 2014. Red Hook: Curran Associates, 2014: 424-432.
[49] IONESCU C, PAPAVA D, OLARU V, et al. Human3.6M: large scale datasets and predictive methods for 3D human sensing in natural environments[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36(7): 1325-1339.
[50] SALIMANS T, GOODFELLOW I, ZAREMBA W, et al. Improved techniques for training GANs[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Dec 5, 2016. Red Hook: Curran Associates, 2016: 2234-2242.
[51] YU H X, WU A C, ZHENG W S. Cross-view asymmetric metric learning for unsupervised person re-identification[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Piscataway:IEEE, 2017: 994-1002.
[52] WANG J Y, ZHU X T, GONG S G, et al. Transferable joint attribute-identity deep learning for unsupervised person re-identification[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 2275-2284.
[53] ZHONG Z, ZHENG L, LI S Z, et al. Generalizing a person retrieval model hetero-and homogeneously[C]//LNCS 11217: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 176-192.