Research Progress of Video Anomaly Detection Technology

doi:10.3778/j.issn.1673-9418.2106117

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (3): 529-540.DOI: 10.3778/j.issn.1673-9418.2106117

• Surveys and Frontiers • Previous Articles Next Articles

Research Progress of Video Anomaly Detection Technology

WU Kaijun¹, HUANG Tao¹^,⁺(), WANG Dicong¹^,², BAI Chenshuai¹, TAO Xiaomiao¹

1. School of Electronic and Information Engineering, Lanzhou Jiaotong University, Lanzhou 730070, China
2. College of Intelligence and Computing, Tianjin University, Tianjin 300350, China

Received:2021-06-21 Revised:2021-08-18 Online:2022-03-01 Published:2021-08-19
About author:WU Kaijun, born in 1978, Ph.D., professor, Ph.D. supervisor. His research interests include video detection and nonlinear dynamics of neurons.
HUANG Tao, born in 1996, M.S. candidate, student member of CCF. His research interest is video anomaly detection.
WANG Dicong, born in 1992, Ph.D. candidate, student member of CCF. His research interests include video object detection and video anomaly detection.
BAI Chenshuai, born in 1998, M.S. candidate, student member of CCF. His research interest is video object detection.
TAO Xiaomiao, born in 1985, Ph.D. candidate, lecturer. Her research interests include video object detection and tracking.
Supported by:
National Natural Science Foundation of China(61966022)

视频异常检测技术研究进展

邬开俊¹, 黄涛¹^,⁺(), 王迪聪¹^,², 白晨帅¹, 陶小苗¹

1.兰州交通大学电子与信息工程学院,兰州 730070
2.天津大学智能与计算学部,天津 300350

通讯作者: + E-mail: 1479987020@qq.com
作者简介:邬开俊（1978—）,男,山东莒南人,博士,教授,博士生导师,主要研究方向为视频检测与神经元的非线性动力学。
黄涛（1996—）,男,重庆彭水人,硕士研究生,CCF 学生会员,主要研究方向为视频的异常检测。
王迪聪(1992—),男,甘肃张掖人,博士研究生,CCF 学生会员,主要研究方向为视频的目标检测、视频的异常检测。
白晨帅（1998—）,男,陕西渭南人,硕士研究生,CCF学生会员,主要研究方向为视频的目标检测。
陶小苗（1985—）,女,山西运城人,博士研究生,讲师,主要研究方向为视频的目标检测和追踪。
基金资助:
国家自然科学基金(61966022)

Abstract

Abstract:

Video anomaly detection refers to the detection and identification of events that deviate from normal behavior, which has a wide range of applications in surveillance video. In this paper, the video anomaly detection algorithm based on deep learning is investigated in depth and summarized comprehensively. Firstly, this paper analyzes the related content of video anomaly detection and the challenges faced by anomaly detection, then introduces and analyzes the related algorithms of video anomaly detection from three aspects: supervised, semi-supervised and unsupervised. The algorithms in three different scenarios are further refined and classified. The algorithms in the supervised scenario are divided into two types: binary classification and multi-classification. The algorithms in the semi-supervised scenario are divided into two types: calculating anomaly scores and clustering discrimination. The algorithms in the unsupervised scenario are divided into two types: reconstruction discrimination and prediction discrimination. The characteristics, advantages and disadvantages of different technologies are analyzed. The commonly used datasets in the field of video anomaly detection and the evaluation criteria of detection performance are introduced, and the performance of current mainstream video anomaly detection algorithms is compared and analyzed. Finally, the future research direction of video anomaly detection algorithm is discussed and prospected.

Key words: deep learning, anomaly detection, supervised, semi-supervised, unsupervised

摘要：

视频异常检测是指对偏离正常行为事件的检测识别,在监控视频中有着广泛的应用。对基于深度学习的视频异常检测算法进行了深入的调查研究和全面的梳理与总结。首先,对视频异常检测相关内容以及异常检测面临的挑战进行了分析;然后,从有监督、半监督和无监督三方面对视频异常检测的相关算法进行了介绍和分析。对三种不同场景下的算法进一步细化分类,将监督场景下的算法划分为二分类和多分类两种方式,将半监督场景下的算法划分为计算异常得分和聚类判别两种方式,将无监督场景下的算法划分为重构判别和预测判别两种方式,并且分析了不同技术的特点和优缺点。介绍了目前在视频异常检测领域常用的数据集,以及检测性能的评估标准,对目前主流的视频异常检测算法性能进行了对比分析。最后,对视频异常检测算法的未来研究方向进行了讨论和展望。

关键词: 深度学习, 异常检测, 有监督, 半监督, 无监督

CLC Number:

TP391

WU Kaijun, HUANG Tao, WANG Dicong, BAI Chenshuai, TAO Xiaomiao. Research Progress of Video Anomaly Detection Technology[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(3): 529-540.

邬开俊, 黄涛, 王迪聪, 白晨帅, 陶小苗. 视频异常检测技术研究进展[J]. 计算机科学与探索, 2022, 16(3): 529-540.

Figures/Tables 3

References 64

[1]	GRUBBS F E. Procedures for detecting outlying observations in samples[J]. Technometrics, 1969, 11(1): 1-21. DOI URL
[2]	MA X X, WU J, XUE S, et al. A comprehensive survey on graph anomaly detection with deep learning[J]. arXiv:2106.07178, 2021.
[3]	SUAREZ J J P, NAVAL JR P C. A survey on deep learning techniques for video anomaly detection[J]. arXiv:2009.14146, 2020.
[4]	BULUSU S, KAILKHURA B, LI B, et al. Anomalous instance detection in deep learning: a survey[J]. arXiv:2003.06979, 2020.
[5]	PANG G, SHEN C, CAO L, et al. Deep learning for anomaly detection: a review[J]. arXiv:2007.02500, 2020.
[6]	王志国, 章毓晋. 监控视频异常检测: 综述[J]. 清华大学学报(自然科学版), 2020, 60(6): 518-529.
	WANG G Z, ZHANG Y J. Anomaly detection in surveillance videos: a survey[J]. Journal of Tsinghua University (Science and Technology), 2020, 60(6): 518-529.
[7]	彭嘉丽, 赵英亮, 王黎明. 基于深度学习的视频异常行为检测研究[J]. 激光与光电子学进展, 2021, 58(6): 43-53.
	PENG J L, ZHAO Y L, WANG L M. Research on video abnormal behavior detection based on deep learning[J]. Laser & Optoelectronics Progress, 2021, 58(6): 43-53.
[8]	CHANDOLA V, BANERJEE A, KUMAR V. Anomaly detection: a survey[J]. ACM Computing Surveys, 2009, 41(3): 1-58.
[9]	张乐. 深度神经网络视频异常目标检测与定位算法研究[D]. 秦皇岛: 燕山大学, 2019.
	ZHANG L. Anomaly detection and localization in video surveillance by deep neural network[D]. Qinghuangdao: Yanshan University, 2019.
[10]	ZHOU S F, SHEN W, ZENG D, et al. Spatial-temporal convolutional neural networks for anomaly detection and localization in crowded scenes[J]. Signal Processing Image Communication, 2016, 47: 358-368. DOI URL
[11]	SABOKROU M, FAYYAZ M, FATHY M, et al. Deep-Cascade: cascading 3D deep neural networks for fast anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Image Processing, 2017, 26(4): 1992-2004. DOI URL
[12]	LIU K, MA H D. Exploring background-bias for anomaly detection in surveillance videos[C]// Proceedings of the 27th ACM International Conference on Multimedia, Nice, Oct 21-25, 2019. New York: ACM, 2019: 1490-1499.
[13]	LANDI F, LANDI F, CUCCHIARA R. Anomaly locality in video surveillance[J]. arXiv:1901.10364, 2019.
[14]	CARREIRA J, ZISSERMAN A. Quo vadis, action recognition? a new model and the kinetics dataset[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 4724-4733.
[15]	HASAN M, CHOI J, NEUMANN J, et al. Learning temporal regularity in video sequences[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 733-742.
[16]	GU X X, CUI J R, ZHU Q. Abnormal crowd behavior detection by using the particle entropy[J]. Optik-International Journal for Light and Electron Optics, 2014, 125(14): 3428-3433. DOI URL
[17]	ANDRADE E L, BLUNSDEN S, FISHER R B. Modelling crowd scenes for event detection[C]// Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China, Aug 20-24, 2006. Washington: IEEE Computer Society, 2006: 175-178.
[18]	MAHADEVAN V, LI W X, BHALODIA V, et al. Anomaly detection in crowded scenes[C]// Proceedings of the 23rd IEEE Conference on Computer Vision and Pattern Recognition, San Francisco, Jun 13-18, 2010. Washington: IEEE Computer Society, 2010: 1975-1981.
[19]	MARKOVITZ A, SHARIR G, FRIEDMAN I, et al. Graph embedded pose clustering for anomaly detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 10536-10544.
[20]	RAMACHANDRA B, JONES M J, VATSAVAI R R. Learning a distance function with a Siamese network to localize anomalies in videos[C]// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 2587-2596.
[21]	CHANG Y P, TU Z G, XIE W, et al. Clustering driven deep autoencoder for video anomaly detection[C]// LNCS 12360: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 329-345.
[22]	ZAHEER M Z, MAHMOOD A, ASTRID M, et al. CLAWS: clustering assisted weakly supervised learning with normalcy suppression for anomalous event detection[C]// LNCS 12367: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 358-376.
[23]	SULTANI W, CHEN C, SHAH M. Real-world anomaly detection in surveillance videos[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscataway: IEEE, 2018: 6479-6488.
[24]	ZHU Y, NEWSAM S. Motion-aware feature for improved video anomaly detection[J]. arXiv:1907.10211, 2019.
[25]	WAN B Y, FANG Y M, XIA X, et al. Weakly supervised video anomaly detection via center-guided discriminative learning[C]// Proceedings of the 2020 IEEE International Conference on Multimedia and Expo, Jul 6-10, 2020. Piscataway: IEEE, 2020: 1-6.
[26]	LIU W, LUO W X, LI Z X, et al. Margin learning embedded prediction for video anomaly detection with a few anomalies[C]// Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 3023-3030.
[27]	ZHONG J X, LI N N, KONG W J, et al. Graph convolutional label noise cleaner: train a plug-and-play action classifier for anomaly detection[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1237-1246.
[28]	ZAHEER M Z, MAHMOOD A, SHIN H, et al. A self-reasoning framework for anomaly detection using video-level labels[J]. IEEE Signal Processing Letters, 2020, 27: 1705-1709. DOI URL
[29]	GONG D, LIU L Q, LE V, et al. Memorizing normality to detect anomaly: memory-augmented deep autoencoder for unsupervised anomaly detection[C]// Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1705-1714.
[30]	IONESCU R T, KHAN F S, GEORGESCU M I, et al. Object-centric auto-encoders and dummy anomalies for abnormal event detection in video[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 7842-7851.
[31]	NGUYEN T N, MEUNIER J. Anomaly detection in video sequence with appearance-motion correspondence[C]// Proceed-ings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1273-1283.
[32]	LIU W, LUO W X, LIAN D Z, et al. Future frame prediction for anomaly detection—a new baseline[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscat- away: IEEE, 2018: 6536-6545.
[33]	RAVANBAKHSH M, SANGINETO E, NABI M, et al. Training adversarial discriminators for cross-channel abnormal event detection in crowds[C]// Proceedings of the 2019 IEEE Winter Conference on Applications of Computer Vision, Waikoloa Village, Jan 7-11, 2019. Piscataway: IEEE, 2019: 1896-1904.
[34]	VU H, NGUYEN T D, LE T, et al. Robust anomaly detection in videos using multilevel representations[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 5216-5223.
[35]	ZAHEER M Z, LEE J H, ASTRID M, et al. Old is gold: redefining the adversarially learned one-class classifier training paradigm[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 14171-14181.
[36]	XIA Y D, ZHANG Y, LIU F Z, et al. Synthesize then compare: detecting failures and anomalies for semantic segmentation[C]// LNCS 12346: Proceedings of the 16th European Confer-ence on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 145-161.
[37]	SABOKROU M, KHALOOEI M, FATHY M, et al. Adver-sarially learned one-class classifier for novelty detection[C]// Proceedings of the 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Salt Lake City,Jun 18-23, 2018. Piscataway: IEEE, 2018: 3379-3388.
[38]	LUO W X, LIU W, GAO S H. A revisit of sparse coding based anomaly detection in stacked RNN framework[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 341-349.
[39]	LU Y W, YU F, REDDY M K K, et al. Few-shot scene-adaptive anomaly detection[C]// LNCS 12350: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 125-141.
[40]	RODRIGUES R, BHARGAVA N, VELMURUGAN R, et al. Multi-timescale trajectory prediction for abnormal human activity detection[C]// Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 2615-2623.
[41]	MORAIS R, LE V, TRAN T, et al. Learning regularity in skeleton trajectories for anomaly detection in videos[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 11996-12004.
[42]	PANG G S, YAN C, SHEN C H, et al. Self-trained deep ordinal regression for end-to-end video anomaly detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 12170-12179.
[43]	DOSHI K, YILMAZ Y. Continual learning for anomaly detection in surveillance videos[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 1025-1034.
[44]	ILG E, MAYER N, SAIKIA T, et al. Flownet 2.0: evolution of optical flow estimation with deep networks[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 1647-1655.
[45]	REDMON J, DIVVALA S K, GIRSHICK R B, et al. You only look once: unified, real-time object detection[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 779-788.
[46]	LU C W, SHI J P, JIA J Y. Abnormal event detection at 150 fps in matlab[C]// Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 2720-2727.
[47]	ADAM A, RIVLIN E, SHIMSHONI I, et al. Robust real-time unusual event detection using multiple fixed-location monitors[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2008, 30(3): 555-560. DOI URL
[48]	SABOKROU M, FAYYAZ M, FATHY M, et al. Deep-anomaly: fully convolutional neural network for fast anomaly detection in crowded scenes[J]. Computer Vision and Image Understanding, 2018, 172: 88-97. DOI URL
[49]	University of Minnesota. UMN[EB/OL]. [2020-08-18]. http://mha.cs.umn.edu/Movies/Crowd-Activity-All.avi.
[50]	LI W X, MAHADEVAN V, VASCONCELOS N. Anomaly detection and localization in crowded scenes[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 36(1): 18-32. DOI URL
[51]	DEL GIORNO A, BAGNELL J A, HEBERT M. A discrim-inative framework for anomaly detection in large videos[C]// LNCS 9909: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 334-349.
[52]	IONESCU R T, SMEUREANU S, ALEXE B, et al. Unmasking the abnormal events in video[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2914-2922.
[53]	HU X M, CHEN Q, YANG L, et al. Abnormal crowd behavior detection and localization based on deep spatial-temporal convolutional neural network[J]. Application Research of Computers, 2020, 37(3): 891-895.
[54]	GONG M G, ZENG H, XIE Y, et al. Local distinguishability aggrandizing network for human anomaly detection[J]. Neural Networks, 2020, 122: 364-373. DOI URL
[55]	ZHANG J G, QING L Y, MIAO J. Temporal convolutional network with complementary inner bag loss for weakly supervised anomaly detection[C]// Proceedings of the 2019 IEEE International Conference on Image Processing, Taiwan, China, Sep 22-25, 2019. Piscataway: IEEE, 2019: 4030-4034.
[56]	ZHANG Y, LU H C, ZHANG L H, et al. Combining motion and appearance cues for anomaly detection[J]. Pattern Recogni-tion, 2016, 51: 443-452.
[57]	LUO W X, LIU W, GAO S H. Remembering history with convolutional LSTM for anomaly detection[C]// Proceedings of the 2017 IEEE International Conference on Multimedia and Expo, Hong Kong, China, Jul 10-14, 2017. Washington:IEEE Computer Society, 2017: 439-444.
[58]	PARK H, NOH J, HAM B. Learning memory-guided normality for anomaly detection[C]// Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 14360-14369.
[59]	SUN Q R, LIU H, HARADA T. Online growing neural gas for anomaly detection in changing surveillance scenes[J]. Pattern Recognition, 2017, 64: 187-201. DOI URL
[60]	TANG Y, ZHAO L, ZHANG S S, et al. Integrating prediction and reconstruction for anomaly detection[J]. Pattern Recogni-tion Letters, 2020, 129: 123-130.
[61]	LEYVA R, SANCHEZ V, LI C T. Video anomaly detection with compact feature sets for online performance[J]. IEEE Transactions on Image Processing, 2017, 26(7): 3463-3478. DOI URL
[62]	RAVANBAKHSH M, NABI M, MOUSAVI H, et al. Plug-and-Play CNN for crowd motion analysis: an application in abnormal event detection[C]// Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, Mar 12-15, 2018. Washington: IEEE Computer Society, 2018: 1689-1698.
[63]	AKHTAR N, MIAN A. Threat of adversarial attacks on deep learning in computer vision: a survey[J]. IEEE Access, 2018, 6: 14410-14430. DOI URL
[64]	SHARMA Y, LE T D, ALZANTOT M. CAAD 2018: generating transferable adversarial examples[J]. arXiv:1810.01268, 2018.

数据集	帧数	标签	分辨率	异常事件
UCSD Ped1	14 000	时间和空间	238×158	汽车、骑自行车等
UCSD Ped2	4 560	时间和空间	360×240	汽车、骑自行车等
UMN Lawn	1 450	时间	320×240	人群逃生
UMN Indoor	4 415	时间	320×240	人群逃生
UMN Plaza	2 145	时间	320×240	人群逃生
CUHK Avenue	30 652	时间和空间	640×360	跑、扔纸、扔书包等
Subway Entrance	72 401	时间	512×384	逃票、徘徊
Subway Exit	136 524	时间	512×384	逃票、徘徊
Shanghai-Tech	317 398	时间和空间	856×480	追逐、突然移动等
UCF-Crime	大约13.8×10⁶	视频级和时间	320×240	逮捕、纵火、攻击等

数据集	帧数	标签	分辨率	异常事件
UCSD Ped1	14 000	时间和空间	238×158	汽车、骑自行车等
UCSD Ped2	4 560	时间和空间	360×240	汽车、骑自行车等
UMN Lawn	1 450	时间	320×240	人群逃生
UMN Indoor	4 415	时间	320×240	人群逃生
UMN Plaza	2 145	时间	320×240	人群逃生
CUHK Avenue	30 652	时间和空间	640×360	跑、扔纸、扔书包等
Subway Entrance	72 401	时间	512×384	逃票、徘徊
Subway Exit	136 524	时间	512×384	逃票、徘徊
Shanghai-Tech	317 398	时间和空间	856×480	追逐、突然移动等
UCF-Crime	大约13.8×10⁶	视频级和时间	320×240	逮捕、纵火、攻击等

模型	监督方式	数据集	是否端到端	方法描述	AUC/%
模型	监督方式	数据集	是否端到端	方法描述	C	U1	U2	S
DSTCNN^[53]	监督	UCSD	否	深度时空卷积神经网络（DSTCNN）提取动作特征并输出正、异常分类概率	—	99.7	99.9	—
LDA-Net^[54]	监督	UCSD	否	YOLO提取的前景人体作为 3D CNN的输入提取行为的时空特征进而分类正、异常行为	—	—	97.9	—
IBL^[55]	半监督	Shanghai-Tech	否	多实例学习 (MIL)定义一种IBL损失来约束弱监督问题的函数空间	—	—	—	82.5
GCN-Anomaly^[27]	半监督	UCSD+Shanghai-Tech	是	图卷积网络校正噪声标签	—	—	93.2	84.4
AR-Net^[25]	半监督	Shanghai-Tech	否	异常回归网络(AR-Net)的框架学习视频级弱监督下的区分特征	—	—	—	91.2
App+motion cues^[56]	半监督	UCSD	否	使用“cut-bin”区分异常运动,使用SVDD外观检测	—	85.0	90.0	—
Conv-AE^[15]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	全卷积自编码器学习运动特征	70.2	81.0	90.0	60.9
ConvLSTM-AE^[57]	无监督	CUHK Avenue	是	ConvLSTM-AE框架来检测外观和外观(运动)的变化	77.0	—	—	—
Sparse coding^[38]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	将稀疏编码和循环神经网络结合	81.7	—	92.2	68.0
Future frame^[32]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	采用U-Net来预测下一帧	85.1	83.1	95.4	72.8
Memory-augmented AE^[29]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	内存增强自编码器（MemAE）查询检索与重构最相关的内存项	83.3	—	94.1	71.2
MNAD^[58]	无监督	CUHK Avenue+Shanghai-Tech	是	存储模块记录正常数据的模式	88.5	—	—	70.5
MLEP^[55]	无监督	CUHK Avenue+Shanghai-Tech	是	ConvLSTM编码时空信息	92.8	—	—	76.8
Unmasking^[52]	无监督	CUHK Avenue	否	对滑窗内部的前后两部分进行人为分类	80.6	—	—	—
Appearance-cGAN^[31]	无监督	CUHK Avenue+UCSD	是	两个子网络Conv-AE解决图像结构问题,U-Net解决运动模式的问题	86.9	—	96.2	—
Online-GNG^[59]	无监督	UCSD	是	GNG 将不同样本映射到不同聚类实现端到端的聚类	—	93.8	94.0	—
Prediction reconstruction^[60]	无监督	CUHK Avenue+UCSD	是	两个U-Net：一个以帧预测的形式工作,另一个尝试重建前一个网络生成的帧	83.7	82.6	96.2	—
Compact features^[61]	无监督	UCSD	否	输入帧分成可变大小的单元结构,提取多重特征并输入多个判别模型进行检测	—	82.0	84.0	—
Plug and play CNN^[62]	无监督	UCSD	是	FCN网络作为预训练模型	—	95.7	88.4	—
Siamese distance learning^[20]	无监督	CUHK Avenue+UCSD	否	孪生卷积神经网络学习一对视频时空区域之间的距离函数	87.2	86.0	94.0	—

模型	监督方式	数据集	是否端到端	方法描述	AUC/%
模型	监督方式	数据集	是否端到端	方法描述	C	U1	U2	S
DSTCNN^[53]	监督	UCSD	否	深度时空卷积神经网络（DSTCNN）提取动作特征并输出正、异常分类概率	—	99.7	99.9	—
LDA-Net^[54]	监督	UCSD	否	YOLO提取的前景人体作为 3D CNN的输入提取行为的时空特征进而分类正、异常行为	—	—	97.9	—
IBL^[55]	半监督	Shanghai-Tech	否	多实例学习 (MIL)定义一种IBL损失来约束弱监督问题的函数空间	—	—	—	82.5
GCN-Anomaly^[27]	半监督	UCSD+Shanghai-Tech	是	图卷积网络校正噪声标签	—	—	93.2	84.4
AR-Net^[25]	半监督	Shanghai-Tech	否	异常回归网络(AR-Net)的框架学习视频级弱监督下的区分特征	—	—	—	91.2
App+motion cues^[56]	半监督	UCSD	否	使用“cut-bin”区分异常运动,使用SVDD外观检测	—	85.0	90.0	—
Conv-AE^[15]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	全卷积自编码器学习运动特征	70.2	81.0	90.0	60.9
ConvLSTM-AE^[57]	无监督	CUHK Avenue	是	ConvLSTM-AE框架来检测外观和外观(运动)的变化	77.0	—	—	—
Sparse coding^[38]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	将稀疏编码和循环神经网络结合	81.7	—	92.2	68.0
Future frame^[32]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	采用U-Net来预测下一帧	85.1	83.1	95.4	72.8
Memory-augmented AE^[29]	无监督	CUHK Avenue+UCSD+Shanghai-Tech	是	内存增强自编码器（MemAE）查询检索与重构最相关的内存项	83.3	—	94.1	71.2
MNAD^[58]	无监督	CUHK Avenue+Shanghai-Tech	是	存储模块记录正常数据的模式	88.5	—	—	70.5
MLEP^[55]	无监督	CUHK Avenue+Shanghai-Tech	是	ConvLSTM编码时空信息	92.8	—	—	76.8
Unmasking^[52]	无监督	CUHK Avenue	否	对滑窗内部的前后两部分进行人为分类	80.6	—	—	—
Appearance-cGAN^[31]	无监督	CUHK Avenue+UCSD	是	两个子网络Conv-AE解决图像结构问题,U-Net解决运动模式的问题	86.9	—	96.2	—
Online-GNG^[59]	无监督	UCSD	是	GNG 将不同样本映射到不同聚类实现端到端的聚类	—	93.8	94.0	—
Prediction reconstruction^[60]	无监督	CUHK Avenue+UCSD	是	两个U-Net：一个以帧预测的形式工作,另一个尝试重建前一个网络生成的帧	83.7	82.6	96.2	—
Compact features^[61]	无监督	UCSD	否	输入帧分成可变大小的单元结构,提取多重特征并输入多个判别模型进行检测	—	82.0	84.0	—
Plug and play CNN^[62]	无监督	UCSD	是	FCN网络作为预训练模型	—	95.7	88.4	—
Siamese distance learning^[20]	无监督	CUHK Avenue+UCSD	否	孪生卷积神经网络学习一对视频时空区域之间的距离函数	87.2	86.0	94.0	—

Research Progress of Video Anomaly Detection Technology

视频异常检测技术研究进展

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 3

References 64

Related Articles 15

Recommended Articles

Metrics

[1]	AN Fengping, LI Xiaowei, CAO Xiang. Medical Image Classification Algorithm Based on Weight Initialization-Sliding Window CNN [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1885-1897.
[2]	ZENG Fanzhi, XU Luqian, ZHOU Yan, ZHOU Yuexia, LIAO Junwei. Review of Knowledge Tracing Model for Intelligent Education [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(8): 1742-1763.
[3]	LIU Yi, LI Mengmeng, ZHENG Qibin, QIN Wei, REN Xiaoguang. Survey on Video Object Tracking Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1504-1515.
[4]	ZHAO Xiaoming, YANG Yijiao, ZHANG Shiqing. Survey of Deep Learning Based Multimodal Emotion Recognition [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1479-1503.
[5]	XIA Hongbin, XIAO Yifei, LIU Yuan. Long Text Generation Adversarial Network Model with Self-Attention Mechanism [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(7): 1603-1610.
[6]	SUN Fangwei, LI Chengyang, XIE Yongqiang, LI Zhongbo, YANG Caidong, QI Jin. Review of Deep Learning Applied to Occluded Object Detection [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1243-1259.
[7]	LIU Yafen, ZHENG Yifeng, JIANG Lingyi, LI Guohe, ZHANG Wenjie. Survey on Pseudo-Labeling Methods in Deep Semi-supervised Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(6): 1279-1290.
[8]	ZHU Zhuangzhuang, ZHOU Zhiping. Detection of Health Data Based on Gaussian Mixture Generative Model [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1128-1135.
[9]	CHENG Weiyue, ZHANG Xueqin, LIN Kezheng, LI Ao. Deep Convolutional Neural Network Algorithm Fusing Global and Local Features [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1146-1154.
[10]	ZHONG Mengyuan, JIANG Lin. Review of Super-Resolution Image Reconstruction Algorithms [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 972-990.
[11]	PEI Lishen, ZHAO Xuezhuan. Survey of Collective Activity Recognition Based on Deep Learning [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 775-790.
[12]	XU Jia, WEI Tingting, YU Ge, HUANG Xinyue, LYU Pin. Review of Question Difficulty Evaluation Approaches [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 734-759.
[13]	ZHU Weijie, CHEN Ying. Micro-expression Recognition Convolutional Network for Dual-stream Temporal-Domain Information Interaction [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(4): 950-958.
[14]	ZHANG Quangui, HU Jiayan, WANG Li. One Class Collaborative Filtering Recommendation Algorithm Coupled with User Common Characteristics [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(3): 637-648.
[15]	LIU Liping, SUN Jian, GAO Shiyan. Overview of Blind Deblurring Methods for Single Image [J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(3): 552-564.