改进的Siamese自适应网络和多特征融合跟踪算法

doi:10.3778/j.issn.1673-9418.2103044

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (11): 2587-2595.DOI: 10.3778/j.issn.1673-9418.2103044

改进的Siamese自适应网络和多特征融合跟踪算法

李睿, 连继荣⁺()

兰州理工大学计算机与通信学院，兰州 730050

收稿日期:2021-03-15 修回日期:2021-06-08 出版日期:2022-11-01 发布日期:2021-06-16
通讯作者: + E-mail: 649094574@qq.com
作者简介:李睿（1971—），女，甘肃秦安人，硕士，教授，主要研究方向为智能信息处理、模式识别、人工智能。
连继荣（1995—），男，甘肃定西人，硕士研究生，主要研究方向为模式识别、人工智能。
基金资助:
国家自然科学基金(61761028);甘肃省重点研发计划-工业类(18YF1GA060)

Improved Siamese Adaptive Network and Multi-feature Fusion Tracking Algorithm

LI Rui, LIAN Jirong⁺()

College of Computer and Communication, Lanzhou University of Technology, Lanzhou 730050, China

Received:2021-03-15 Revised:2021-06-08 Online:2022-11-01 Published:2021-06-16
About author:LI Rui, born in 1971, M.S., professor. Her research interests include intelligent information processing, pattern recognition and artificial intelligence.
LIAN Jirong, born in 1995, M.S. candidate. His research interests include pattern recognition and artificial intelligence.
Supported by:
National Natural Science Foundation of China(61761028);Key Research and Development Program of Gansu Province-Industrial(18YF1GA060)

摘要/Abstract

摘要：

针对当前目标跟踪领域中跟踪精确度和跟踪速度难以平衡的问题，例如基于相关滤波实现的跟踪器能够以很高的速度运行，但跟踪准确性极低；基于深度学习实现的跟踪器能够实现较高的跟踪准确性，但跟踪速度较低。在此基础上，提出一种改进的Siamese自适应网络和多特征融合目标跟踪算法。首先在Siamese网络每个分支上同时构建AlexNet网络和改进的ResNet网络，用于特征提取。其次通过端到端的方式同时进行训练，将跟踪问题分解为分类每个位置标签和回归边界框子问题。最后对浅层特征和深层特征进行自适应选择，并基于多特征融合进行目标识别和定位。将提出的算法与现有的一些跟踪器在目标跟踪标准数据集上进行测试。实验结果表明, 提出的算法能够在确保跟踪速度的同时实现较高的跟踪精确度和成功率。同时，在光照变化、形变、背景杂波等复杂情况下，算法具有较强的鲁棒性。

关键词: 目标跟踪, Siamese网络, 特征融合, 尺度自适应, ResNet网络

Abstract:

Aiming at the problem that tracking accuracy and tracking speed are difficult to balance in the current target tracking field. For example, a tracker based on correlation filtering can run at a very high speed, but the tracking accuracy is extremely low; a tracker based on deep learning can achieve high tracking accuracy, but the tracking speed is extremely low. On this basis, an improved Siamese adaptive network and multi-feature fusion target tracking algorithm are proposed. Firstly, the AlexNet network and the improved ResNet network are constructed on each branch of the Siamese network at the same time for feature extraction. Secondly, through end-to-end training at the same time, the tracking problem is decomposed into sub-problems of classifying each position label and returning to the bounding box. Finally, the shallow features and deep features are selected adaptively, and the target recognition and location are carried out based on multi-feature fusion. The proposed algorithm and some existing trackers are tested on the target tracking standard dataset. Experimental results show that the proposed algorithm can achieve high target tracking accuracy and success rate while ensuring tracking speed. At the same time, the algorithm has strong robustness in complex situations such as illumination changes, deformations, and background clutter.

Key words: object tracking, Siamese network, feature fusion, scale adaptation, ResNet network

中图分类号:

TP391

李睿, 连继荣. 改进的Siamese自适应网络和多特征融合跟踪算法[J]. 计算机科学与探索, 2022, 16(11): 2587-2595.

LI Rui, LIAN Jirong. Improved Siamese Adaptive Network and Multi-feature Fusion Tracking Algorithm[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(11): 2587-2595.

图/表 11

图1 SiamFC网络结构

Fig.1 SiamFC network structure

图2 改进的Siamese自适应网络结构

Fig.2 Improved Siamese adaptive network structure

图3 残差单元结构

Fig.3 Residual unit structure

图4 改进的残差单元结构

Fig.4 Improved residual unit structure

图5 跟踪效果比较

Fig.5 Tracking performance comparison

图6 精度和成功率曲线图

Fig.6 Accuracy graph and success rate graph

图7 形变影响下的精度和成功率曲线图

Fig.7 Accuracy graph and success rate graph drawn under influence of deformation

图8 背景杂波影响下的精度和成功率曲线图

Fig.8 Accuracy graph and success rate graph drawn under influence of background clutter

图9 遮挡影响下的精度和成功率曲线图

Fig.9 Accuracy graph and success rate graph drawn under influence of occlusion

表1 提出的算法与已有跟踪器性能对比

Table 1 Performance comparison between proposed algorithm and existing trackers

算法名称	成功率	精度	速度/（frame/s）
Ours	0.804	0.831	63.82
Struck	0.559	0.656	14.45
LOT	0.413	0.522	35.62
TLD	0.521	0.608	45.53
CT	0.348	0.406	20.59
SMS	0.185	0.337	89.57
MTT	0.445	0.475	49.83
CSK	0.443	0.545	94.66

表2 算法内部比较

Table 2 Algorithm internal comparison

名称	成功率	精度	速度/（frame/s）
N₁	0.586	0.642	80.35
N₂	0.713	0.686	68.61
N₃	0.786	0.813	62.54
F₁	0.467	0.516	98.96
F₂	0.424	0.498	25.31
F₃	0.658	0.579	56.32
M₁	0.864	0.882	48.69
M₂	0.753	0.806	92.73

参考文献 35

[1]	陈晨, 邓赵红, 高艳丽, 等. 多模糊核融合的单目标跟踪算法[J]. 计算机科学与探索, 2020, 14(5): 848-860. DOI
	CHEN C, DENG Z H, GAO Y L, et al. Single target trac-king algorithm based on multi-fuzzy kernel fusion[J]. Jour-nal of Frontiers of Computer Science and Technology, 2020, 14(5): 848-860.
[2]	卢湖川, 李佩霞, 王栋. 目标跟踪算法综述[J]. 模式识别与人工智能, 2018, 31(1): 61-76. DOI
	LU H C, LI P X, WANG D. Visual object tracking: a survey[J]. Pattern Recognition and Artificial Intelligence, 2018, 31(1): 61-76. DOI
[3]	YANG H, SHAO L, ZHENG F, et al. Recent advances and trends in visual tracking: a review[J]. Neurocomputing, 2011, 74(18): 3823-3831. DOI URL
[4]	KALAL Z, MIKOLAJCZYK K, MATAS J. Tracking learning-detection[J]. IEEE Transactions on Pattern Analysis and Ma-chine Intelligence, 2012, 34(7): 1409-1422.
[5]	LI B, YAN J, WU W, et al. High performance visual tracking with siamese region proposal network[C]// Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Washington: IEEE Computer Society, 2018: 8971-8980.
[6]	BERTINETTO L, VALMADRE J, HENRIQUES J F, et al. Fully-convolutional siamese networks for object tracking[C]// LNCS 9914: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 8-10 and 15-16, 2016. Cham: Springer, 2016: 850-865.
[7]	ZHU Z, WANG Q, LI B, et al. Distractor-aware siamese networks for visual object tracking[C]// LNCS 11213: Procee-dings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 103-119.
[8]	LI B, WU W, WANG Q, et al. SiamRPN++: evolution of siamese visual tracking with very deep networks[C]// Pro-ceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Pis-cataway: IEEE, 2019: 4282-4291.
[9]	GUO D, WANG J, CUI Y, et al. SiamCAR: siamese fully convolutional classification and regression for visual trac-king[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 6268-6276.
[10]	SMEULDERS A W M, CHU D M, CUCCHIARA R, et al. Visual tracking: an experimental survey[J]. IEEE Transac-tions on Pattern Analysis and Machine Intelligence, 2013, 36(7): 1442-1468.
[11]	汪洪桥, 蔡艳宁, 孙富春, 等. 多尺度核方法的自适应序列学习及应用[J]. 模式识别与人工智能, 2011, 24(1): 72-81.
	WANG H Q, CAI Y N, SUN F C, et al. Adaptive sequence learning and applications for multi-scale kernel method[J]. Pattern Recognition and Artificial Intelligence, 2011, 24(1):72-81.
[12]	王玲, 王家沛, 王鹏, 等. 融合注意力机制的孪生网络目标跟踪算法研究[J]. 计算机工程与应用, 2021, 57(8): 169-174. DOI
	WANG L, WANG J P, WANG P, et al. Research on target tracking algorithm of twin network fusion attention mecha-nism[J]. Computer Engineering and Applications, 2021, 57(8): 169-174. DOI
[13]	单玉刚, 胡卫国. 尺度方向自适应视觉目标跟踪方法综述[J]. 计算机工程与应用, 2020, 56(9): 13-23. DOI
	SHAN Y G, HU W G. A survey of adaptive vision target tracking methods in scale and direction[J]. Computer Enginee-ring and Applications, 2020, 56(9): 13-23.
[14]	HENRIQUES J F, CASEIRO R, MARTINS P, et al. High-speed tracking with kernelized correlation filters[J]. IEEE Tran-sactions on Pattern Analysis and Machine Intelligence, 2015, 37(3): 583-596.
[15]	POSSEGGER H, MAUTHNER T, BISCHOF H. In defense of color-based model-free tracking[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Re-cognition, Boston, Jun 7-12, 2015. Washington: IEEE Com-puter Society, 2015: 2113-2120.
[16]	VALMADRE J, BERTINETTO L, HENRIQUES J F, et al. End-to-end representation learning for correlation filter based tracking[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 5000-5008.
[17]	GAO J Y, ZHANG T Z, XU C S. Graph convolutional trac-king[C]// Proceedings of the 2019 IEEE Conference on Com-puter Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4649-4659.
[18]	ZHANG L, VARADARAJAN J, SUGANTHAN P N, et al. Robust visual tracking using oblique random forests[C]// Pro-ceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washing-ton: IEEE Computer Society, 2017: 5825-5834.
[19]	DANELLJAN M, BHAT G, KHAN F S, et al. ATOM: accurate tracking by overlap maximization[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pat-tern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4660-4669.
[20]	MA C, HUANG J B, YANG X K, et al. Hierarchical con-volutional features for visual tracking[C]// Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer So-ciety, 2015: 3074-3082.
[21]	BOLME D S, BEVERIDGE J R, DRAPER B A, et al. Visual object tracking using adaptive correlation filters[C]// Procee-dings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, Jun 13-18, 2010. Washing-ton: IEEE Computer Society, 2010: 2544-2550.
[22]	LI F, YAO Y J, LI P H, et al. Integrating boundary and cen-ter correlation filters for visual tracking with aspect ratio variation[C]// Proceedings of the 2017 IEEE International Con-ference on Computer Vision Workshops, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2001-2009.
[23]	ZHANG J, MA S, SCLAROFF S. MEEM: robust tracking via multiple experts using entropy minimization[C]// LNCS 8694: Proceedings of the 13th European Conference on Com-puter Vision, Zurich, Sep 6-12, 2014. Cham: Springer, 2014: 188-203.
[24]	KRIZHEVSKY A, SUTSKEVER I, HINTON G E. Image-Net classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90. DOI URL
[25]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual lear-ning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition,Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer So-ciety, 2016: 770-778.
[26]	KRISTAN M, LEONARDIS A, MATAS J, et al. The visual object tracking vot2017 challenge results[C]// Proceedings of the 2017 IEEE International Conference on Computer Vi-sion, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 1949-1972.
[27]	WU Y, LIM J, YANG M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 2015, 37(9): 1834-1848.
[28]	RUSSAKOVSKY O, DENG J, SU H, et al. ImageNet large scale visual recognition challenge[J]. International Journal of Computer Vision, 2015, 115(3): 211-252. DOI URL
[29]	HARE S, GOLODETZ S, SAFFARI A, et al. Struck: struc-tured output tracking with kernels[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 65(5): 2096-2109.
[30]	ORON S, BAR-HILLEL A, LEVI D, et al. Locally orderless tracking[J]. International Journal of Computer Vision, 2015, 111(2): 213-228. DOI URL
[31]	KALAL Z, MATAS J, MIKOLAJCZYK K. P-N learning: bootstrapping binary classifiers by structural constraints[C]// Proceedings of the 23rd IEEE Conference on Computer Vi-sion and Pattern Recognition, San Francisco, Jun 13-18, 2010. Washington: IEEE Computer Society, 2010: 49-56.
[32]	ZHANG K H, ZHANG L, YANG M H. Real-time compre-ssive tracking[C]// LNCS 7574: Proceedings of the 12th Euro-pean Conference on Computer Vision, Florence, Oct 7-13, 2012. Berlin, Heidelberg: Springer, 2012: 864-877.
[33]	COLLINS R T. Mean-shift blob tracking through scale space[C]// Proceedings of the 2003 IEEE Computer Society Con-ference on Computer Vision and Pattern Recognition, Ma-dison, Jun 16-22, 2003. Washington: IEEE Computer So-ciety, 2003: 234-240.
[34]	ZHANG T Z, GHANEM B, LIU S, et al. Robust visual trac-king via multi-task sparse learning[C]// Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Re-cognition, Providence, Jun 16-21, 2012. Washington: IEEE Computer Society, 2012: 2042-2049.
[35]	HENRIQUES J F, RUI C, MARTINS P, et al. Exploiting the circulant structure of tracking-by-detection with kernels[C]// LNCS 7575: Proceedings of the 12th European Conference on Computer Vision, Florence, Oct 7-13, 2012. Berlin, Hei-delberg: Springer, 2012: 702-715.

改进的Siamese自适应网络和多特征融合跟踪算法

Improved Siamese Adaptive Network and Multi-feature Fusion Tracking Algorithm

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 11

参考文献 35

相关文章 15

编辑推荐

Metrics

[1]	杨政, 邓赵红, 罗晓清, 顾鑫, 王士同. 利用ELM-AE和迁移表征学习构建的目标跟踪系统[J]. 计算机科学与探索, 2022, 16(7): 1633-1648.
[2]	彭豪, 李晓明. 多尺度选择金字塔网络的小样本目标检测算法[J]. 计算机科学与探索, 2022, 16(7): 1649-1660.
[3]	刘艺, 李蒙蒙, 郑奇斌, 秦伟, 任小广. 视频目标跟踪算法综述[J]. 计算机科学与探索, 2022, 16(7): 1504-1515.
[4]	赵运基, 范存良, 张新良. 融合多特征和通道感知的目标跟踪算法[J]. 计算机科学与探索, 2022, 16(6): 1417-1428.
[5]	李运寰, 闻继伟, 彭力. 高帧率的轻量级孪生网络目标跟踪[J]. 计算机科学与探索, 2022, 16(6): 1405-1416.
[6]	程卫月, 张雪琴, 林克正, 李骜. 融合全局与局部特征的深度卷积神经网络算法[J]. 计算机科学与探索, 2022, 16(5): 1146-1154.
[7]	赵鹏飞, 谢林柏, 彭力. 融合注意力机制的深层次小目标检测算法[J]. 计算机科学与探索, 2022, 16(4): 927-937.
[8]	包广斌, 李港乐, 王国雄. 面向多模态情感分析的双模态交互注意力[J]. 计算机科学与探索, 2022, 16(4): 909-916.
[9]	程世龙, 谢林柏, 彭力. 梯度导向的通道选择目标跟踪算法[J]. 计算机科学与探索, 2022, 16(3): 649-660.
[10]	王燕妮, 余丽仙. 注意力与多尺度有效融合的SSD目标检测算法[J]. 计算机科学与探索, 2022, 16(2): 438-447.
[11]	陈璠, 彭力. 异构分支关联特征融合的行人重识别[J]. 计算机科学与探索, 2022, 16(11): 2609-2618.
[12]	余鹰, 潘诚, 朱慧琳, 钱进, 汤洪. 融合通道与空间注意力的编解码人群计数算法[J]. 计算机科学与探索, 2022, 16(11): 2547-2556.
[13]	李青援, 邓赵红, 罗晓清, 顾鑫, 王士同. 注意力与跨尺度融合的SSD目标检测算法[J]. 计算机科学与探索, 2022, 16(11): 2575-2586.
[14]	钱伍, 王国中, 李国平. 改进YOLOv5的交通灯实时检测鲁棒算法[J]. 计算机科学与探索, 2022, 16(1): 231-241.
[15]	李志欣, 陈圣嘉, 周韬, 马慧芳. 协同级联网络和对抗网络的目标检测[J]. 计算机科学与探索, 2022, 16(1): 217-230.