Detection Optimized Multi-Bernoulli Algorithm for Visual Multi-target Tracking

doi:10.3778/j.issn.1673-9418.1908068

Abstract

Abstract:

For the problems of the Multi-Bernoulli filter (MBF) in the multi-target tracking filed, such as the newborn targets cannot be detected, the tracking accuracy decreases or the targets are underestimated when the targets are occluded, and the underestimated targets cannot be associated with their previous tracks when they are redetected. The YOLOv3 method is introduced to predetect the targets under the framework of MBF, and then the predetected targets are presented by the convolution features and their similarity matrix are calculated between the adjacent frames. The detection strategy is proposed to realize the discrimination of newborn target. The reiden-tifiable strategy is proposed to achieve the continuous estimation of underestimated targets. In addition, when the template is updated, the high-confidence detection is integrated, and the occluded target processing mechanism is proposed to effectively improve the accuracy of target tracking. Finally, the proposed algorithm is tested on some challenging video sequences from the public standard dataset. The results show that the proposed algorithm has a good performance on detecting the newborn targets and the underestimated targets, realizing continuous tracking of visual multi-target trajectory.

Key words: multi-Bernoulli filter, tracking-by-detection, newborn target, similarity matrix

摘要：

针对多伯努利滤波方法在多目标跟踪时，难以检测新生目标，且当目标出现互相遮挡等干扰时，跟踪精度下降，甚至出现目标漏跟，以及当漏跟目标被重新跟踪后，与之前运动轨迹难以关联等问题，在多伯努利滤波框架下，引入YOLOv3检测算法，并采用卷积特征对目标进行描述，计算相邻帧目标的相似度矩阵，设计新生目标识别和漏跟目标的重识别策略，以实现对目标新生判别和漏跟目标的连续估计；此外，在模板更新时，融合高置信度检测框，提出遮挡目标处理机制，有效提高目标跟踪精度。最后，采用标准数据集中具有挑战性的视频序列进行算法测试，结果表明，提出算法能有效识别新生目标和漏跟目标，实现对视频多目标轨迹的连续跟踪。

关键词: 多伯努利滤波, 检测跟踪, 目标新生, 相似度矩阵

YANG Jinlong, CHENG Xiaoxue, MIAO Jiani, ZHANG Guangnan. Detection Optimized Multi-Bernoulli Algorithm for Visual Multi-target Tracking[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(10): 1762-1775.

杨金龙，程小雪，缪佳妮，张光南. 检测优化的多伯努利视频多目标跟踪算法[J]. 计算机科学与探索, 2020, 14(10): 1762-1775.

References

[1] Luo W, Xing J, Milan A, et al. Multiple object tracking: a literature review[J]. arXiv:1409.7618, 2017.
[2] Kim C, Li F X, Ciptadi A, et al. Multiple hypothesis tracking revisited[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 4696-4704.
[3] Ba H X, Cao L, He X Y, et al. Modified joint probabilistic data association with classification-aided for multitarget tracking[J]. Journal of Systems Engineering and Electronics, 2008, 19(3): 434-439.
[4] Mahler R P S. Representing rules as random sets, II: iterated rules[J]. International Journal of Intelligent Systems, 2015, 11(8): 583-610.
[5] Mahler R P S. Multitarget Bayes filtering via first-order multitarget moments[J]. IEEE Transactions on Aerospace and Electronic Systems, 2003, 39(4): 1152-1178.
[6] Mahler R P S. PHD filters of higher order in target number[J]. IEEE Transactions on Aerospace and Electronic Systems, 2007, 43(4): 1523-1543.
[7] Vo B T, Vo B N, Cantoni A. Analytic implementations of the cardinalized probability hypothesis density filter[J]. IEEE Transactions on Signal Processing, 2007, 55(7): 3553-3567.
[8] Hoseinnezhad R, Vo B N, Vo B T, et al. Visual tracking of numerous targets via multi-Bernoulli filtering of image data[J]. Pattern Recognition, 2012, 45(10): 3625-3635.
[9] Vo B N, Vo B T, Pham N T, et al. Joint detection and estima-tion of multiple objects from image observations[J]. IEEE Transactions on Signal Processing, 2010, 58(10): 5129-5141.
[10] Vo B T, Vo B N, Cantoni A. The cardinality balanced multi-target multi-bernoulli filter and its implementations[J]. IEEE Transactions on Signal Processing, 2009, 57(2): 409-423.
[11] Vo B T, Vo B N, Hoseinnezhad R, et al. Robust multi-Bernoulli filtering[J]. IEEE Journal of Selected Topics in Signal Processing, 2013, 7(3): 399-409.
[12] Saeidi M, Ahmadi A. Deep learning based on CNN for pedes-trian detection: an overview and analysis[C]//Proceedings of the 9th International Symposium on Telecommunications, Tehran, Dec 17-19, 2018. Piscataway: IEEE, 2018: 108-112.
[13] Fu Z, Naqvi S M, Chambers J A. Collaborative detector fusion of data-driven PHD filter for online multiple human tracking[C]//Proceedings of the 21st International Conference on Information Fusion, Cambridge, Jul 10-13, 2018. Pisca-taway: IEEE, 2018: 1976-1981.
[14] Breitenstein M D, Reichlin F, Leibe B, et al. Online multi-person tracking-by-detection from a single, uncalibrated camera[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2011, 33(9): 1820-1833.
[15] Vo B N, Singh S, Doucet A. Sequential Monte Carlo methods for multitarget filtering with random finite sets[J]. IEEE Transactions on Aerospace and Electronic Systems, 2005, 51(4): 1224-1245.
[16] Hoseinnezhad R, Vo B N, Vo B T. Visual tracking in back-ground subtracted image sequences via multi-Bernoulli filter-ing[J]. IEEE Transactions on Signal Processing, 2013, 61(2): 392-397.
[17] Rathnayake T, Gostar A K, Hoseinnezhad R, et al. Labeled multi-Bernoulli track-before-detect for multi-target tracking in video[C]//Proceedings of the 18th International Confer-ence on Information Fusion, Washington, Jul 6-9, 2015. Pis-cataway: IEEE, 2015: 1353-1358.
[18] Kim D Y. Online multi-object tracking via labeled random finite set with appearance learning[C]//Proceedings of the 2017 International Conference on Control, Automation and Information Sciences, Chiang Mai, Oct 31-Nov 1, 2017. Piscataway: IEEE, 2017: 181-186.
[19] Ristic B, Clark D, Vo B N, et al. Adaptive target birth inten-sity for PHD and CPHD filters[J]. IEEE Transactions on Aerospace and Electronic Systems, 2012, 48(2): 1656-1668.
[20] Si W J, Wang L W, Qu Z Y. A measurement-driven adaptive probability hypothesis density filter for multitarget tracking[J]. Chinese Journal of Aeronautics, 2015, 28(6): 1689-1698.
[21] Redmon J, Farhadi A. YOLOv3: an incremental improve-ment[J]. arXiv:1804.02767, 2018.
[22] Redmon J, Divvala S, Girshick R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the 2016 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Wash-ington: IEEE Computer Society, 2016: 779-788.
[23] Redmon J, Farhadi A. YOLO900: better, faster, stronger[C]// Proceedings of the 2017 IEEE Computer Society Confer-ence on Computer Vision and Pattern Recognition, Hono-lulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 6517-6525.
[24] Liu W, Anguelov D, Erhan D, et al. SSD: single shot multi-box detector[C]//LNCS 9905: Proceedings of the 14th Eur-opean Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Berlin, Heidelberg: Springer, 2016: 21-37.
[25] Zhang K H, Liu Q S, Wu Y, et al. Robust visual tracking via convolutional networks without training[J]. IEEE Transac-tions on Image Processing, 2015, 25(4): 1779-1792.
[26] Hartigan J A, Wong M A. Algorithm AS 136: a [K-means]clustering algorithm[J]. Journal of the Royal Statistical Society, 1979, 28(1): 100-108.
[27] Wu Y, Lim J, Yang M H. Object tracking benchmark[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015, 37(9): 1834-1848.
[28] Dataset of European commission funded context aware vision using image-based active recognition[EB/OL]. (2019-04-25) [2019-06-04]. http://homepages.inf.ed.ac.uk/rbf/CAVIARD-ATA.
[29] Yang J L, Tang Y, Zhang G N. Visual multiobject tracking using convolution feature and multibernoulli filter[J]. Journal of Frontiers of Computer Science and Technology, 2019, 13(11): 1945-1957.杨金龙, 汤玉, 张光南. 卷积特征多伯努利视频多目标跟踪算法[J]. 计算机科学与探索, 2019, 13(11): 1945-1957.
[30] Bochinski E, Eiselein V, Sikora T. High-speed tracking-by-detection without using image information[C]//Proceedings of the 14th IEEE International Conference on Advanced Video & Signal Based Surveillance, Lecce, Aug 29-Sep 1, 2017. Washington: IEEE Computer Society, 2017: 1-6.
[31] Ristic B, Vo B N, Clark D, et al. A metric for performance evaluation of multitarget tracking algorithms[J]. IEEE Tran-sactions on Signal Processing, 2011, 59(7): 3452-3457.
[32] Bernardin K, Stiefelhagen R. Evaluating multiple object track-ing performance: the CLEAR MOT metrics[J]. EURASIP Journal on Image and Video Processing, 2008, 1: 1-10.