Detection Optimized Labeled Multi-Bernoulli Algorithm for Visual Multi-target Tracking

doi:10.3778/j.issn.1673-9418.2109110

Abstract

Abstract: In a video multi-target tracking algorithm combining a detector with a tracker, the quality of detector affects the performance of the whole tracking algorithm. The missing and false detection will lead to the missing and false tracking of target, increase the fragmentation trajectory and increase the number of identity tag transformation. In order to solve these problems, this paper further optimizes the tracking algorithm in the framework of labeled multi-Bernoulli filter, designs a new measurement driven newborn target recognition method to capture newborn targets more quickly and accurately, designs a new target recognition method which can maintain the label invariance in a short time and reduce the fragmentation trajectory and label jumping, and introduces a new template selection strategy to avoid polluting the template by adding the occluded target to the template. Considering the labeled multi-Bernoulli filter is an online reasoning algorithm, parallelization is adopted to speed up the operation efficiency of the algorithm. The result shows that the proposed algorithm can effectively solve the problems of label jumping and inaccurate tracking by target occlusion. It is tested on the challenging MOT17 dataset and has good tracking effect compared with other relevant filtering methods.

Key words: labeled multi-Bernoulli filter, tracking-by-detection, feature extraction, target reidentification

摘要： 在检测器与跟踪器结合的视频多目标跟踪算法中，检测器好坏将直接影响整个跟踪算法性能，尤其是检测器的漏检以及误检，会导致目标的漏跟以及误跟，增加碎片化轨迹以及身份标签变换次数增加的问题。针对这些问题，在标签多伯努利的滤波框架下，设计了新的量测驱动新生目标识别方法以更快速精准地捕获新生目标。设计了目标重识别方法，结合标签多伯努利算法能够在短时间内维持标签的不变性，减少了碎片化轨迹及标签跳变数。引入新的模板选取策略，以避免将被遮挡的目标加入到模板中污染模板。考虑到标签多伯努利滤波为在线推理算法，采用了并行化加快算法的运算效率。结果表明，在标签多伯努利的框架下，提出算法能够有效解决标签跳变以及目标被遮挡无法准确跟踪的问题，在具有挑战性的MOT17数据集上进行测试，与其他相关滤波方法进行比较，具有不错的跟踪效果。

关键词: 标签多伯努利滤波, 检测跟踪, 特征提取, 目标重识别

JIANG Lingyun, YANG Jinlong. Detection Optimized Labeled Multi-Bernoulli Algorithm for Visual Multi-target Tracking[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(6): 1343-1358.

蒋凌云, 杨金龙. 检测优化的标签多伯努利视频多目标跟踪算法[J]. 计算机科学与探索, 2023, 17(6): 1343-1358.

References

[1] 夏晨木, 滕奇志, 卿粼波, 等. 岩石三维图像裂缝提取方法[J]. 计算机工程与应用, 2018, 54(17): 186-191.
XIA C M, TENG Q Z, QING L B, et al. Crack recognition of 3D rock images[J]. Computer Engineering and Applications, 2018, 54(17): 186-191.
[2] 李震霄, 孙伟, 刘明明, 等. 交通监控场景中的车辆检测与跟踪算法研究[J]. 计算机工程与应用, 2021, 57(8): 103-111.
LI Z X, SUN W, LIU M M, et al. Research on vehicle detection and tracking algorithms in traffic monitoring scenes[J]. Computer Engineering and Applications, 2021, 57(8): 103-111.
[3] 牛通, 卿粼波, 许盛宇, 等. 基于深度学习的分层关联多行人跟踪[J]. 计算机工程与应用, 2021, 57(8): 96-102.
NIU T, QING L B, XU S Y, et al. Multiple target tracking using hierarchical data association based on deep learning[J]. Computer Engineering and Applications, 2021, 57(8): 96-102.
[4] 鄂贵, 王永雄. 基于R-FCN框架的多候选关联在线多目标跟踪[J]. 光电工程, 2020, 47(1): 29-37.
E G, WANG Y X. Multi-candidate association online multi-target tracking based on R-FCN framework[J]. Opto-Electronic Engineering, 2020, 47(1): 29-37.
[5] REUTER S, VO B T, VO B N, et al. The labeled multi-Bernoulli filter[J]. IEEE Transactions on Signal Processing, 2014, 62(12): 3246-3260.
[6] VO B T, VO B N. Labeled random finite sets and multi-object conjugate priors[J]. IEEE Transactions on Signal Processing, 2013, 61(13): 3460-3475.
[7] VO B N, VO B T, PHUNG D Q. Labeled random finite sets and the Bayes multi-target tracking filter[J]. IEEE Transactions on Signal Processing, 2014, 62(24): 6554-6567.
[8] YIN J J, ZHANG J Q, ZHAO J. The Gaussian particle multi-target multi-Bernoulli filter[C]//Proceedings of the 2nd International Conference on Advanced Computer Control, Shenyang, Mar 27-29, 2010. Piscataway: IEEE, 2010: 556-560.
[9] FU Z Y, FENG P M, ANGELINI F, et al. Particle PHD filter based multiple human tracking using online group-structured dictionary learning[J]. IEEE Access, 2018, 6: 14764-14778.
[10] FU Z Y, ANGELINI F, NAQVI S M, et al. GM-PHD filter based online multiple human tracking using deep discriminative correlation matching[C]//Proceedings of the 2018 IEEE International Conference on Acoustics, Speech and Signal Processing, Calgary, Apr 15-20, 2018. Piscataway: IEEE, 2018: 4299-4303.
[11] FU Z Y, ANGELINI F, CHAMBERS J, et al. Multi-level cooperative fusion of GM-PHD filters for online multiple human tracking[J]. IEEE Transactions on Multimedia, 2019, 21(9): 2277-2291.
[12] 杨金龙, 汤玉, 张光南. 卷积特征多伯努利视频多目标跟踪算法[J]. 计算机科学与探索, 2019, 13(11): 1945-1957.
YANG J L, TANG Y, ZHANG G N. Visual multi-object tracking using convolution feature and multi-Bernoulli filter[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(11): 1945-1957.
[13] HENSCHEL R, LEALT L, CREMERS D, et al. Fusion of head and full-body detectors for multi-object tracking[C]//Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 1428-1437.
[14] 杨金龙, 程小雪, 缪佳妮, 等. 检测优化的多伯努利视频多目标跟踪算法[J]. 计算机科学与探索, 2020, 14(10): 1762-1775.
YANG J L, CHENG X X, MIAO J N, et al. Detection optimized multi-Bernoulli algorithm for visual multi-target tracking[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(10): 1762-1775.
[15] KUTSCHBACH T, BOCHINSKI E, EISELEIN V, et al. Sequential sensor fusion combining probability hypothesis density and kernelized correlation filters for multi-object tracking in video data[C]//Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance, Lecce, Aug 29-Sep 1, 2017. Washington: IEEE Computer Society, 2017: 1-5.
[16] FU Z Y, NAQVI S M, CHAMBERS J A, et al. Collaborative detector fusion of data-driven PHD filter for online multiple human tracking[C]//Proceedings of the 21st International Conference on Information Fusion, Cambridge, Jul 10-13, 2018. Piscataway: IEEE, 2018: 1976-1981.
[17] HOSEINNEZHAD R, VO B N, VO B T, et al. Bayesian integration of audio and visual information for multi-target tracking using a CB-MeMBer filter[C]//Proceedings of the 2011 IEEE International Conference on Acoustics, Speech, and Signal Processing, Prague, May 22-27, 2011. Piscataway: IEEE, 2011: 2300-2303.
[18] KIM D Y. Online multi-object tracking via labeled random finite set with appearance learning[C]//Proceedings of the 2017 International Conference on Control, Automation and Information Sciences, Chiang Mai, Oct 31-Nov 1, 2017. Piscataway: IEEE, 2017: 181-186.
[19] WANG Z D, ZHENG L, LIU Y X, et al. Towards real-time multi-object tracking[C]//LNCS 12356: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 107-122.
[20] LAND E H, MCCANN J J. Lightness and retinex theory[J]. Journal of the Optical Society of America, 1971, 61(1): 1-11.
[21] BAISA N L. Occlusion-robust online multi-object visual tracking using a GM-PHD filter with a CNN-based reidentification[J]. arXiv:1912.05949, 2019.
[22] LIAO S C, HU Y, ZHU X Y, et al. Person re-identification by local maximal occurrence representation and metric learning[C]//Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 2197-2206.
[23] JOBSON D J, RAHMAN Z U, WOODELL G A. A multiscale retinex for bridging the gap between color images and the human observation of scenes[J]. IEEE Transactions on Image Processing, 1997, 6(7): 965-976.
[24] JOBSON D J, RAHMAN Z U, WOODELL G A. Properties and performance of a center/surround retinex[J]. IEEE Transactions on Image Processing, 1997, 6(3): 451-462.
[25] SANCHEZ-MATILLA R, CAVALLARO A. Motion prediction for first-person vision multi-object tracking[C]//LNCS 12538: Proceedings of the 16th ECCV Workshops on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 485-499.
[26] BEWLEY A, GE Z Y, OTT L, et al. Simple online and realtime tracking[C]//Proceedings of the 2016 IEEE International Conference on Image Processing, Phoenix, Sep 25-28, 2016. Piscataway: IEEE, 2016: 3464-3468.
[27] BOCHINSKI E, EISELEIN V, SIKORA T. High-speed tracking-by-detection without using image information[C]//Proceedings of the 14th IEEE International Conference on Advanced Video and Signal Based Surveillance, Lecce, Aug 29-Sep 1, 2017. Washington: IEEE Computer Society, 2017: 1-6.
[28] RISTANI E, SOLERA F, ZOU R, et al. performance measures and a data set for multi-target, multi-camera tracking[C]//LNCS 9914: Proceedings of the 14th ECCV Workshops on Computer Vision, Amsterdam, Oct 8-10, 15-16, 2016. Cham: Springer, 2016: 17-35.