Double Residual Network Recognition Method for Falling Abnormal Behavior

doi:10.3778/j.issn.1673-9418.1906054

Abstract

Abstract:

In the abnormal behavior monitoring, due to the complicated situation such as monitoring angles of view, human body postures and scenes, it is easy to cause vanishing gradient and over-fitting by directly adding 3D con-volutional neural network layers to extract effective visual features, which reduces the action recognition rate. To solve these problems, this paper proposes a fall recognition method based on the double residual convolutional network. By nesting the residual network in the residual network, the double residual network fully integrates shallow and deep visual features and alleviates the impact of the vanishing gradient, and makes the performance of residual network improved. Finally, multiple cameras fall dataset (MCFD) and UR fall dataset (URFD) are tested and evaluated by the 5-fold cross-validation method. The results show that the performance is better than some fall recognition methods based on 3D convolutional network (C3D), 3D residual network (3D-Resnet), Pseudo-3D residual network (P3D), and (2+1)D residual network (R(2+1)D), which verifies the effectiveness of the double residual network model for improving the abnormal behavior recognition.

Key words: fall recognition, residual network, vanishing gradient, action recognition

摘要：

在异常行为监控中，由于监控视角、人体姿态和场景等复杂的情况，直接通过增加3D卷积神经网络层数来提取有效的视觉特征，容易导致卷积模型发生梯度消失和过拟合，从而降低了行为识别率。针对上述问题，提出了一种基于双重残差卷积网络的跌倒识别方法，通过在残差网络中嵌套残差网络，充分融合了浅层和深层视觉特征，缓解了模型训练时梯度消失问题带来的影响，从而使模型性能得到了提升。最后采用5折交叉验证方法在多相机跌倒数据集（MCFD）和热舒夫大学跌倒数据集（URFD）上进行了测试评估，结果表明双重残差网络性能优于三维卷积网络（C3D）、三维残差网络（3D-Resnet）、伪三维残差网络（P3D）和2+1维残差网络（R(2+1)D）识别方法，从而验证了双重残差网络模型对提高异常行为识别效果的有效性。

关键词: 跌倒识别, 残差网络, 梯度消失, 行为识别

WANG Xinwen, XIE Linbo, PENG Li. Double Residual Network Recognition Method for Falling Abnormal Behavior[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(9): 1580-1589.

王新文，谢林柏，彭力. 跌倒异常行为的双重残差网络识别方法[J]. 计算机科学与探索, 2020, 14(9): 1580-1589.

References

[1] United Nations. Department of Economic and Social Affairs. Population Division. World population prospects: the 2015 revision[EB/OL]. (2015-12-09)[2019-06-01]. http://www.un.org/en/development/desa/population/theme/ageing/WPA2015.shtml.
[2] World Health Organization. Falls: key facts[EB/OL]. (2018-01-16)[2019-06-01]. http://www.who.int/news-room/fact-sheets/detail/falls.
[3] Mubashir M, Shao L, Seed L. A survey on fall detection: prin-ciples and approaches[J]. Neurocomputing, 2013, 100(2): 144-152.
[4] Liang W J, Zhang Y H, Jing H, et al. Research on fall detec-tion method based on SVM[J]. Measurement & Control Technology, 2014, 33(9): 33-35.梁维杰, 张应红, 景晖, 等. 基于支持向量机的跌倒检测方法研究[J]. 测控技术, 2014, 33(9): 33-35.
[5] Kong X B, Lin M, Tomiyama H. Fall detection for elderly persons using a depth camera[C]//Proceedings of the 2017 International Conference on Advanced Mechatronic Systems, Xiamen, Dec 6-9, 2017. Piscataway: IEEE, 2018: 269-273.
[6] Min W D, Cui H, Rao H, et al. Detection of human falls on furniture using scene analysis based on deep learning and activity characteristics[J]. IEEE Access, 2018, 6: 9324-9335.
[7] Shi X, Zhang T. Design of a wearable fall detection device[J]. Chinese Journal of Scientific Instrument, 2012, 33(3): 575-580.石欣, 张涛. 一种可穿戴式跌倒检测装置设计[J]. 仪器仪表学报, 2012, 33(3): 575-580.
[8] Mirmahboub B, Samavi S, Karimi N, et al. Automatic mono-cular system for human fall detection based on variations in silhouette area[J]. IEEE Transactions on Biomedical Engi-neering, 2013, 60(2): 427-436.
[9] Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intel-ligence, 2017, 39(6): 1137-1149.
[10] Krizhevsky A, Sutskever I, Hinton G E. ImageNet classifi-cation with deep convolutional neural networks[C]//Proceed-ings of the 25th International Conference on Neural Infor-mation Processing Systems, Doha, Nov 12-15, 2012. New York: ACM, 2012: 1097-1105.
[11] He K M, Zhang X Y, Ren S Q. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Con-ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[12] Simonyan K, Zisserman A. Two-stream convolutional net-works for action recognition in videos[J]. Computational Linguistics, 2014, 1(4): 568-576.
[13] Tran D, Bourdev L, Fergus R, et al. Learning spatiotem-poral features with 3D convolutional networks[C]//Proceed-ings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 13-16, 2015. Piscataway: IEEE, 2015: 4489-4497.
[14] Qiu Z F, Yao T, Mei T. Learning spatio-temporal represent-ation with pseudo-3D residual networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 24-27, 2017. Piscataway: IEEE, 2017: 5534-5542.
[15] Hara K, Kataoka H, Satoh Y. Learning spatio-temporal fea-tures with 3D residual networks for action recognition[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision Workshops, Venice, Oct 28-29, 2017. Piscat-away: IEEE, 2017: 3154-3160.
[16] Tran D, Wang H, Torresani L, et al. A closer look at spatio-temporal convolutions for action recognition[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 6450-6459.
[17] Auvinet E, Rougier C, Meunier J, et al. Multiple cameras fall dataset[EB/OL]. [2019-06-01]. http://www.iro.umontreal.ca/~labimage/Dataset/.
[18] Kwolek B, Kepski M. Human fall detection on embedded platform using depth maps and wireless accelerometer[J]. Computer Methods & Programs in Biomedicine, 2014, 117(3): 489-501.
[19] Soomro K, Zamir A R, Shah M. UCF101: a dataset of 101 human action classes from videos in the wild[EB/OL]. [2019-06-01]. http://crcv.ucf.edu/data/UCF101.php.
[20] Wang L M, Xiong Y J, Wang Z, et al. Towards good practices for very deep two-stream convnets[J]. arXiv:1507.02159, 2015.