视频异常检测技术研究进展

doi:10.3778/j.issn.1673-9418.2106117

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (3): 529-540.DOI: 10.3778/j.issn.1673-9418.2106117

视频异常检测技术研究进展

邬开俊¹, 黄涛¹^,⁺(), 王迪聪¹^,², 白晨帅¹, 陶小苗¹

1.兰州交通大学电子与信息工程学院,兰州 730070
2.天津大学智能与计算学部,天津 300350

收稿日期:2021-06-21 修回日期:2021-08-18 出版日期:2022-03-01 发布日期:2021-08-19
通讯作者: + E-mail: 1479987020@qq.com
作者简介:邬开俊（1978—）,男,山东莒南人,博士,教授,博士生导师,主要研究方向为视频检测与神经元的非线性动力学。
黄涛（1996—）,男,重庆彭水人,硕士研究生,CCF 学生会员,主要研究方向为视频的异常检测。
王迪聪(1992—),男,甘肃张掖人,博士研究生,CCF 学生会员,主要研究方向为视频的目标检测、视频的异常检测。
白晨帅（1998—）,男,陕西渭南人,硕士研究生,CCF学生会员,主要研究方向为视频的目标检测。
陶小苗（1985—）,女,山西运城人,博士研究生,讲师,主要研究方向为视频的目标检测和追踪。
基金资助:
国家自然科学基金(61966022)

Research Progress of Video Anomaly Detection Technology

WU Kaijun¹, HUANG Tao¹^,⁺(), WANG Dicong¹^,², BAI Chenshuai¹, TAO Xiaomiao¹

1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. College of Intelligence and Computing, Tianjin University, Tianjin 300350, China

Received:2021-06-21 Revised:2021-08-18 Online:2022-03-01 Published:2021-08-19
About author:WU Kaijun, born in 1978, Ph.D., professor, Ph.D. supervisor. His research interests include video detection and nonlinear dynamics of neurons.
HUANG Tao, born in 1996, M.S. candidate, student member of CCF. His research interest is video anomaly detection.
WANG Dicong, born in 1992, Ph.D. candidate, student member of CCF. His research interests include video object detection and video anomaly detection.
BAI Chenshuai, born in 1998, M.S. candidate, student member of CCF. His research interest is video object detection.
TAO Xiaomiao, born in 1985, Ph.D. candidate, lecturer. Her research interests include video object detection and tracking.
Supported by:
National Natural Science Foundation of China(61966022)

摘要/Abstract

摘要：

视频异常检测是指对偏离正常行为事件的检测识别,在监控视频中有着广泛的应用。对基于深度学习的视频异常检测算法进行了深入的调查研究和全面的梳理与总结。首先,对视频异常检测相关内容以及异常检测面临的挑战进行了分析;然后,从有监督、半监督和无监督三方面对视频异常检测的相关算法进行了介绍和分析。对三种不同场景下的算法进一步细化分类,将监督场景下的算法划分为二分类和多分类两种方式,将半监督场景下的算法划分为计算异常得分和聚类判别两种方式,将无监督场景下的算法划分为重构判别和预测判别两种方式,并且分析了不同技术的特点和优缺点。介绍了目前在视频异常检测领域常用的数据集,以及检测性能的评估标准,对目前主流的视频异常检测算法性能进行了对比分析。最后,对视频异常检测算法的未来研究方向进行了讨论和展望。

关键词: 深度学习, 异常检测, 有监督, 半监督, 无监督

Abstract:

Video anomaly detection refers to the detection and identification of events that deviate from normal behavior, which has a wide range of applications in surveillance video. In this paper, the video anomaly detection algorithm based on deep learning is investigated in depth and summarized comprehensively. Firstly, this paper analyzes the related content of video anomaly detection and the challenges faced by anomaly detection, then introduces and analyzes the related algorithms of video anomaly detection from three aspects: supervised, semi-supervised and unsupervised. The algorithms in three different scenarios are further refined and classified. The algorithms in the supervised scenario are divided into two types: binary classification and multi-classification. The algorithms in the semi-supervised scenario are divided into two types: calculating anomaly scores and clustering discrimination. The algorithms in the unsupervised scenario are divided into two types: reconstruction discrimination and prediction discrimination. The characteristics, advantages and disadvantages of different technologies are analyzed. The commonly used datasets in the field of video anomaly detection and the evaluation criteria of detection performance are introduced, and the performance of current mainstream video anomaly detection algorithms is compared and analyzed. Finally, the future research direction of video anomaly detection algorithm is discussed and prospected.

Key words: deep learning, anomaly detection, supervised, semi-supervised, unsupervised

中图分类号:

TP391

邬开俊, 黄涛, 王迪聪, 白晨帅, 陶小苗. 视频异常检测技术研究进展[J]. 计算机科学与探索, 2022, 16(3): 529-540.

WU Kaijun, HUANG Tao, WANG Dicong, BAI Chenshuai, TAO Xiaomiao. Research Progress of Video Anomaly Detection Technology[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(3): 529-540.

图/表 3

参考文献 64

[1]	GRUBBS F E. Procedures for detecting outlying observations in samples[J]. Technometrics, 1969, 11(1): 1-21. DOI URL
[2]	MA X X, WU J, XUE S, et al. A comprehensive survey on graph anomaly detection with deep learning[J]. arXiv:2106.07178, 2021.
[3]	SUAREZ J J P, NAVAL JR P C. A survey on deep learning techniques for video anomaly detection[J]. arXiv:2009.14146, 2020.
[4]	BULUSU S, KAILKHURA B, LI B, et al. Anomalous instance detection in deep learning: a survey[J]. arXiv:2003.06979, 2020.
[5]	PANG G, SHEN C, CAO L, et al. Deep learning for anomaly detection: a review[J]. arXiv:2007.02500, 2020.
[6]	王志国, 章毓晋. 监控视频异常检测: 综述[J]. 清华大学学报(自然科学版), 2020, 60(6): 518-529.
	WANG G Z, ZHANG Y J. Anomaly detection in surveillance videos: a survey[J]. Journal of Tsinghua University (Science and Technology), 2020, 60(6): 518-529.
[7]	彭嘉丽, 赵英亮, 王黎明. 基于深度学习的视频异常行为检测研究[J]. 激光与光电子学进展, 2021, 58(6): 43-53.
	PENG J L, ZHAO Y L, WANG L M. Research on video abnormal behavior detection based on deep learning[J]. Laser & Optoelectronics Progress, 2021, 58(6): 43-53.
[8]	CHANDOLA V, BANERJEE A, KUMAR V. Anomaly detection: a survey[J]. ACM Computing Surveys, 2009, 41(3): 1-58.
[9]	张乐. 深度神经网络视频异常目标检测与定位算法研究[D]. 秦皇岛: 燕山大学, 2019.
	ZHANG L. Anomaly detection and localization in video surveillance by deep neural network[D]. Qinghuangdao: Yanshan University, 2019.
[10]	ZHOU S F, SHEN W, ZENG D, et al. Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes[J]. Signal Processing Image Communication, 2016, 47: 358-368. DOI URL
[11]	SABOKROU M, FAYYAZ M, FATHY M, et al. Deep-Cascade: cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1992-2004. DOI URL
[12]	LIU K, MA H D. Exploring background-bias for anomaly detection in surveillance videos[C]// Proceedings of the 27th ACM International Conference on Multimedia, Nice, Oct 21-25, 2019. New York: ACM, 2019: 1490-1499.
[13]	LANDI F, LANDI F, CUCCHIARA R. Anomaly locality in video surveillance[J]. arXiv:1901.10364, 2019.
[14]	CARREIRA J, ZISSERMAN A. Quo vadis, action recognition? a new model and the kinetics dataset[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 4724-4733.
[15]	HASAN M, CHOI J, NEUMANN J, et al. Learning temporal regularity in video sequences[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 733-742.
[16]	GU X X, CUI J R, ZHU Q. Abnormal crowd behavior detection by using the particle entropy[J]. Optik-International Journal for Light and Electron Optics, 2014, 125(14): 3428-3433. DOI URL
[17]	ANDRADE E L, BLUNSDEN S, FISHER R B. Modelling crowd scenes for event detection[C]// Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China, Aug 20-24, 2006. Washington: IEEE Computer Society, 2006: 175-178.
[18]	MAHADEVAN V, LI W X, BHALODIA V, et al. Anomaly detection in crowded scenes[C]// Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, Jun 13-18, 2010. Washington: IEEE Computer Society, 2010: 1975-1981.
[19]	MARKOVITZ A, SHARIR G, FRIEDMAN I, et al. Graph embedded pose clustering for anomaly detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 10536-10544.
[20]	RAMACHANDRA B, JONES M J, VATSAVAI R R. Learning a distance function with a Siamese network to localize anomalies in videos[C]// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 2587-2596.
[21]	CHANG Y P, TU Z G, XIE W, et al. Clustering driven deep autoencoder for video anomaly detection[C]// LNCS 12360: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 329-345.
[22]	ZAHEER M Z, MAHMOOD A, ASTRID M, et al. CLAWS: clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection[C]// LNCS 12367: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 358-376.
[23]	SULTANI W, CHEN C, SHAH M. Real-world anomaly detection in surveillance videos[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscataway: IEEE, 2018: 6479-6488.
[24]	ZHU Y, NEWSAM S. Motion-aware feature for improved video anomaly detection[J]. arXiv:1907.10211, 2019.
[25]	WAN B Y, FANG Y M, XIA X, et al. Weakly supervised video anomaly detection via center-guided discriminative learning[C]// Proceedings of the 2020 IEEE International Conference on Multimedia and Expo, Jul 6-10, 2020. Piscataway: IEEE, 2020: 1-6.
[26]	LIU W, LUO W X, LI Z X, et al. Margin learning embedded prediction for video anomaly detection with a few anomalies[C]// Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 3023-3030.
[27]	ZHONG J X, LI N N, KONG W J, et al. Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1237-1246.
[28]	ZAHEER M Z, MAHMOOD A, SHIN H, et al. A self-reasoning framework for anomaly detection using video-level labels[J]. IEEE Signal Processing Letters, 2020, 27: 1705-1709. DOI URL
[29]	GONG D, LIU L Q, LE V, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1705-1714.
[30]	IONESCU R T, KHAN F S, GEORGESCU M I, et al. Object-centric auto-encoders and dummy anomalies for abnormal event detection in video[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 7842-7851.
[31]	NGUYEN T N, MEUNIER J. Anomaly detection in video sequence with appearance-motion correspondence[C]// Proceed-ings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1273-1283.
[32]	LIU W, LUO W X, LIAN D Z, et al. Future frame prediction for anomaly detection—a new baseline[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscat- away: IEEE, 2018: 6536-6545.
[33]	RAVANBAKHSH M, SANGINETO E, NABI M, et al. Training adversarial discriminators for cross-channel abnormal event detection in crowds[C]// Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, Waikoloa Village, Jan 7-11, 2019. Piscataway: IEEE, 2019: 1896-1904.
[34]	VU H, NGUYEN T D, LE T, et al. Robust anomaly detection in videos using multilevel representations[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 5216-5223.
[35]	ZAHEER M Z, LEE J H, ASTRID M, et al. Old is gold: redefining the adversarially learned one-class classifier training paradigm[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 14171-14181.
[36]	XIA Y D, ZHANG Y, LIU F Z, et al. Synthesize then compare: detecting failures and anomalies for semantic segmentation[C]// LNCS 12346: Proceedings of the 16th European Confer-ence on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 145-161.
[37]	SABOKROU M, KHALOOEI M, FATHY M, et al. Adver-sarially learned one-class classifier for novelty detection[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City,Jun 18-23, 2018. Piscataway: IEEE, 2018: 3379-3388.
[38]	LUO W X, LIU W, GAO S H. A revisit of sparse coding based anomaly detection in stacked RNN framework[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 341-349.
[39]	LU Y W, YU F, REDDY M K K, et al. Few-shot scene-adaptive anomaly detection[C]// LNCS 12350: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 125-141.
[40]	RODRIGUES R, BHARGAVA N, VELMURUGAN R, et al. Multi-timescale trajectory prediction for abnormal human activity detection[C]// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 2615-2623.
[41]	MORAIS R, LE V, TRAN T, et al. Learning regularity in skeleton trajectories for anomaly detection in videos[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 11996-12004.
[42]	PANG G S, YAN C, SHEN C H, et al. Self-trained deep ordinal regression for end-to-end video anomaly detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 12170-12179.
[43]	DOSHI K, YILMAZ Y. Continual learning for anomaly detection in surveillance videos[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 1025-1034.
[44]	ILG E, MAYER N, SAIKIA T, et al. Flownet 2.0: evolution of optical flow estimation with deep networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 1647-1655.
[45]	REDMON J, DIVVALA S K, GIRSHICK R B, et al. You only look once: unified, real-time object detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 779-788.
[46]	LU C W, SHI J P, JIA J Y. Abnormal event detection at 150 fps in matlab[C]// Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 2720-2727.
[47]	ADAM A, RIVLIN E, SHIMSHONI I, et al. Robust real-time unusual event detection using multiple fixed-location monitors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(3): 555-560. DOI URL
[48]	SABOKROU M, FAYYAZ M, FATHY M, et al. Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes[J]. Computer Vision and Image Understanding, 2018, 172: 88-97. DOI URL
[49]	University of Minnesota. UMN[EB/OL]. [2020-08-18]. http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi.
[50]	LI W X, MAHADEVAN V, VASCONCELOS N. Anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(1): 18-32. DOI URL
[51]	DEL GIORNO A, BAGNELL J A, HEBERT M. A discrim-inative framework for anomaly detection in large videos[C]// LNCS 9909: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 334-349.
[52]	IONESCU R T, SMEUREANU S, ALEXE B, et al. Unmasking the abnormal events in video[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2914-2922.
[53]	HU X M, CHEN Q, YANG L, et al. Abnormal crowd behavior detection and localization based on deep spatial-temporal convolutional neural network[J]. Application Research of Computers, 2020, 37(3): 891-895.
[54]	GONG M G, ZENG H, XIE Y, et al. Local distinguishability aggrandizing network for human anomaly detection[J]. Neural Networks, 2020, 122: 364-373. DOI URL
[55]	ZHANG J G, QING L Y, MIAO J. Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection[C]// Proceedings of the 2019 IEEE International Conference on Image Processing, Taiwan, China, Sep 22-25, 2019. Piscataway: IEEE, 2019: 4030-4034.
[56]	ZHANG Y, LU H C, ZHANG L H, et al. Combining motion and appearance cues for anomaly detection[J]. Pattern Recogni-tion, 2016, 51: 443-452.
[57]	LUO W X, LIU W, GAO S H. Remembering history with convolutional LSTM for anomaly detection[C]// Proceedings of the 2017 IEEE International Conference on Multimedia and Expo, Hong Kong, China, Jul 10-14, 2017. Washington:IEEE Computer Society, 2017: 439-444.
[58]	PARK H, NOH J, HAM B. Learning memory-guided normality for anomaly detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 14360-14369.
[59]	SUN Q R, LIU H, HARADA T. Online growing neural gas for anomaly detection in changing surveillance scenes[J]. Pattern Recognition, 2017, 64: 187-201. DOI URL
[60]	TANG Y, ZHAO L, ZHANG S S, et al. Integrating prediction and reconstruction for anomaly detection[J]. Pattern Recogni-tion Letters, 2020, 129: 123-130.
[61]	LEYVA R, SANCHEZ V, LI C T. Video anomaly detection with compact feature sets for online performance[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3463-3478. DOI URL
[62]	RAVANBAKHSH M, NABI M, MOUSAVI H, et al. Plug-and-Play CNN for crowd motion analysis: an application in abnormal event detection[C]// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, Mar 12-15, 2018. Washington: IEEE Computer Society, 2018: 1689-1698.
[63]	AKHTAR N, MIAN A. Threat of adversarial attacks on deep learning in computer vision: a survey[J]. IEEE Access, 2018, 6: 14410-14430. DOI URL
[64]	SHARMA Y, LE T D, ALZANTOT M. CAAD 2018: generating transferable adversarial examples[J]. arXiv:1810.01268, 2018.

数据集	帧数	标签	分辨率	异常事件
UCSD Ped1	14 000	时间和空间	238×158	汽车、骑自行车等
UCSD Ped2	4 560	时间和空间	360×240	汽车、骑自行车等
UMN Lawn	1 450	时间	320×240	人群逃生
UMN Indoor	4 415	时间	320×240	人群逃生
UMN Plaza	2 145	时间	320×240	人群逃生
CUHK Avenue	30 652	时间和空间	640×360	跑、扔纸、扔书包等
Subway Entrance	72 401	时间	512×384	逃票、徘徊
Subway Exit	136 524	时间	512×384	逃票、徘徊
Shanghai-Tech	317 398	时间和空间	856×480	追逐、突然移动等
UCF-Crime	大约13.8×10⁶	视频级和时间	320×240	逮捕、纵火、攻击等

数据集	帧数	标签	分辨率	异常事件
UCSD Ped1	14 000	时间和空间	238×158	汽车、骑自行车等
UCSD Ped2	4 560	时间和空间	360×240	汽车、骑自行车等
UMN Lawn	1 450	时间	320×240	人群逃生
UMN Indoor	4 415	时间	320×240	人群逃生
UMN Plaza	2 145	时间	320×240	人群逃生
CUHK Avenue	30 652	时间和空间	640×360	跑、扔纸、扔书包等
Subway Entrance	72 401	时间	512×384	逃票、徘徊
Subway Exit	136 524	时间	512×384	逃票、徘徊
Shanghai-Tech	317 398	时间和空间	856×480	追逐、突然移动等
UCF-Crime	大约13.8×10⁶	视频级和时间	320×240	逮捕、纵火、攻击等

模型	监督方式	数据集	是否端到端	方法描述	AUC/%
模型	监督方式	数据集	是否端到端	方法描述	C	U1	U2	S
DSTCNN^[53]	监督	UCSD	否	深度时空卷积神经网络（DSTCNN）提取动作特征并输出正、异常分类概率	—	99.7	99.9	—
LDA-Net^[54]	监督	UCSD	否	YOLO提取的前景人体作为 3D CNN的输入提取行为的时空特征进而分类正、异常行为	—	—	97.9	—
IBL^[55]	半监督	Shanghai-Tech	否	多实例学习 (MIL)定义一种IBL损失来约束弱监督问题的函数空间	—	—	—	82.5
GCN-Anomaly^[27]	半监督	UCSD+Shanghai-Tech	是	图卷积网络校正噪声标签	—	—	93.2	84.4
AR-Net^[25]	半监督	Shanghai-Tech	否	异常回归网络(AR-Net)的框架学习视频级弱监督下的区分特征	—	—	—	91.2
App+motion cues^[56]	半监督	UCSD	否	使用“cut-bin”区分异常运动,使用SVDD外观检测	—	85.0	90.0	—
Conv-AE^[15]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	全卷积自编码器学习运动特征	70.2	81.0	90.0	60.9
ConvLSTM-AE^[57]	无监督	CUHK Avenue	是	ConvLSTM-AE框架来检测外观和外观(运动)的变化	77.0	—	—	—
Sparse coding^[38]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	将稀疏编码和循环神经网络结合	81.7	—	92.2	68.0
Future frame^[32]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	采用U-Net来预测下一帧	85.1	83.1	95.4	72.8
Memory-augmented AE^[29]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	内存增强自编码器（MemAE）查询检索与重构最相关的内存项	83.3	—	94.1	71.2
MNAD^[58]	无监督	CUHK Avenue+Shanghai-Tech	是	存储模块记录正常数据的模式	88.5	—	—	70.5
MLEP^[55]	无监督	CUHK Avenue+Shanghai-Tech	是	ConvLSTM编码时空信息	92.8	—	—	76.8
Unmasking^[52]	无监督	CUHK Avenue	否	对滑窗内部的前后两部分进行人为分类	80.6	—	—	—
Appearance-cGAN^[31]	无监督	CUHK Avenue+UCSD	是	两个子网络Conv-AE解决图像结构问题,U-Net解决运动模式的问题	86.9	—	96.2	—
Online-GNG^[59]	无监督	UCSD	是	GNG 将不同样本映射到不同聚类实现端到端的聚类	—	93.8	94.0	—
Prediction reconstruction^[60]	无监督	CUHK Avenue+UCSD	是	两个U-Net：一个以帧预测的形式工作,另一个尝试重建前一个网络生成的帧	83.7	82.6	96.2	—
Compact features^[61]	无监督	UCSD	否	输入帧分成可变大小的单元结构,提取多重特征并输入多个判别模型进行检测	—	82.0	84.0	—
Plug and play CNN^[62]	无监督	UCSD	是	FCN网络作为预训练模型	—	95.7	88.4	—
Siamese distance learning^[20]	无监督	CUHK Avenue+UCSD	否	孪生卷积神经网络学习一对视频时空区域之间的距离函数	87.2	86.0	94.0	—

模型	监督方式	数据集	是否端到端	方法描述	AUC/%
模型	监督方式	数据集	是否端到端	方法描述	C	U1	U2	S
DSTCNN^[53]	监督	UCSD	否	深度时空卷积神经网络（DSTCNN）提取动作特征并输出正、异常分类概率	—	99.7	99.9	—
LDA-Net^[54]	监督	UCSD	否	YOLO提取的前景人体作为 3D CNN的输入提取行为的时空特征进而分类正、异常行为	—	—	97.9	—
IBL^[55]	半监督	Shanghai-Tech	否	多实例学习 (MIL)定义一种IBL损失来约束弱监督问题的函数空间	—	—	—	82.5
GCN-Anomaly^[27]	半监督	UCSD+Shanghai-Tech	是	图卷积网络校正噪声标签	—	—	93.2	84.4
AR-Net^[25]	半监督	Shanghai-Tech	否	异常回归网络(AR-Net)的框架学习视频级弱监督下的区分特征	—	—	—	91.2
App+motion cues^[56]	半监督	UCSD	否	使用“cut-bin”区分异常运动,使用SVDD外观检测	—	85.0	90.0	—
Conv-AE^[15]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	全卷积自编码器学习运动特征	70.2	81.0	90.0	60.9
ConvLSTM-AE^[57]	无监督	CUHK Avenue	是	ConvLSTM-AE框架来检测外观和外观(运动)的变化	77.0	—	—	—
Sparse coding^[38]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	将稀疏编码和循环神经网络结合	81.7	—	92.2	68.0
Future frame^[32]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	采用U-Net来预测下一帧	85.1	83.1	95.4	72.8
Memory-augmented AE^[29]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	内存增强自编码器（MemAE）查询检索与重构最相关的内存项	83.3	—	94.1	71.2
MNAD^[58]	无监督	CUHK Avenue+Shanghai-Tech	是	存储模块记录正常数据的模式	88.5	—	—	70.5
MLEP^[55]	无监督	CUHK Avenue+Shanghai-Tech	是	ConvLSTM编码时空信息	92.8	—	—	76.8
Unmasking^[52]	无监督	CUHK Avenue	否	对滑窗内部的前后两部分进行人为分类	80.6	—	—	—
Appearance-cGAN^[31]	无监督	CUHK Avenue+UCSD	是	两个子网络Conv-AE解决图像结构问题,U-Net解决运动模式的问题	86.9	—	96.2	—
Online-GNG^[59]	无监督	UCSD	是	GNG 将不同样本映射到不同聚类实现端到端的聚类	—	93.8	94.0	—
Prediction reconstruction^[60]	无监督	CUHK Avenue+UCSD	是	两个U-Net：一个以帧预测的形式工作,另一个尝试重建前一个网络生成的帧	83.7	82.6	96.2	—
Compact features^[61]	无监督	UCSD	否	输入帧分成可变大小的单元结构,提取多重特征并输入多个判别模型进行检测	—	82.0	84.0	—
Plug and play CNN^[62]	无监督	UCSD	是	FCN网络作为预训练模型	—	95.7	88.4	—
Siamese distance learning^[20]	无监督	CUHK Avenue+UCSD	否	孪生卷积神经网络学习一对视频时空区域之间的距离函数	87.2	86.0	94.0	—

视频异常检测技术研究进展

Research Progress of Video Anomaly Detection Technology

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 3

参考文献 64

相关文章 15

编辑推荐

Metrics

[1]	安凤平, 李晓薇, 曹翔. 权重初始化-滑动窗口CNN的医学图像分类[J]. 计算机科学与探索, 2022, 16(8): 1885-1897.
[2]	曾凡智, 许露倩, 周燕, 周月霞, 廖俊玮. 面向智慧教育的知识追踪模型研究综述[J]. 计算机科学与探索, 2022, 16(8): 1742-1763.
[3]	刘艺, 李蒙蒙, 郑奇斌, 秦伟, 任小广. 视频目标跟踪算法综述[J]. 计算机科学与探索, 2022, 16(7): 1504-1515.
[4]	赵小明, 杨轶娇, 张石清. 面向深度学习的多模态情感识别研究进展[J]. 计算机科学与探索, 2022, 16(7): 1479-1503.
[5]	夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610.
[6]	孙方伟, 李承阳, 谢永强, 李忠博, 杨才东, 齐锦. 深度学习应用于遮挡目标检测算法综述[J]. 计算机科学与探索, 2022, 16(6): 1243-1259.
[7]	刘雅芬, 郑艺峰, 江铃燚, 李国和, 张文杰. 深度半监督学习中伪标签方法综述[J]. 计算机科学与探索, 2022, 16(6): 1279-1290.
[8]	朱壮壮, 周治平. 高斯混合生成模型检测健康数据异常[J]. 计算机科学与探索, 2022, 16(5): 1128-1135.
[9]	程卫月, 张雪琴, 林克正, 李骜. 融合全局与局部特征的深度卷积神经网络算法[J]. 计算机科学与探索, 2022, 16(5): 1146-1154.
[10]	钟梦圆, 姜麟. 超分辨率图像重建算法综述[J]. 计算机科学与探索, 2022, 16(5): 972-990.
[11]	裴利沈, 赵雪专. 群体行为识别深度学习方法研究综述[J]. 计算机科学与探索, 2022, 16(4): 775-790.
[12]	许嘉, 韦婷婷, 于戈, 黄欣悦, 吕品. 题目难度评估方法研究综述[J]. 计算机科学与探索, 2022, 16(4): 734-759.
[13]	朱伟杰, 陈莹. 双流时间域信息交互的微表情识别卷积网络[J]. 计算机科学与探索, 2022, 16(4): 950-958.
[14]	张全贵, 胡嘉燕, 王丽. 耦合用户公共特征的单类协同过滤推荐算法[J]. 计算机科学与探索, 2022, 16(3): 637-648.
[15]	刘利平, 孙建, 高世妍. 单图像盲去模糊方法概述[J]. 计算机科学与探索, 2022, 16(3): 552-564.