Algorithm for Real-Time Smoking Detection Based on Deep Learning

doi:10.3778/j.issn.1673-9418.2009069

Abstract

Abstract:

In public, smoking behavior not only causes pathological harm to human health, but also exerts danger of fire hazards, etc. For the health and safety considerations, this paper designs a real-time smoking detection model based on deep learning for airports, gas stations, chemical warehouses and other places where smoking is strictly prohibited. This model uses a convolutional neural network to process the input frame from the video stream captured by the webcam. Through the process of image feature extraction, feature fusion, target classification and target posi-tioning, the coordinate of the cigarette is located, and then the smoking behaviors can be found out. Common object detection algorithms are not ideal for small target objects and the detection speed needs to be improved. This paper designs a series of convolutional neural network modules to reduce the amount of model parameters and pick up the inference speed to meet real-time requirements as well as improving the accuracy of small target object (cigarette) detection. This paper also comes up with some training skills to make the model more robust. Due to the lack of relevant dataset, this paper produces a dataset related to smoking behaviors. Through comparative experiments, it is proven that the algorithm proposed in this paper has better detection effects on proposed dataset and some public datasets.

Key words: computer vision, small object detection, real-time, smoking detection, robustness

摘要：

在公共场所内吸烟，不仅对自身、他人身体健康造成潜在的危害，还存在造成火灾等现象的隐患。因此，出于健康和安全方面的考虑，为机场、加油站、化工仓库等严禁吸烟的场所，设计了一种基于深度学习的能快速发现和警告吸烟行为的检测模型。该模型使用卷积神经网络对摄像头所拍摄的视频流输入帧进行处理，经过图像特征提取、特征融合、目标分类以及目标定位等过程，定位烟头的位置，进而判断出吸烟行为。常见的目标检测算法针对小目标物体检测效果不甚理想，检测速度亦有待提高。通过设计的一系列卷积神经网络模块，不但减少了模型计算量，加快了推演速度，满足实时性要求，而且提高了小目标物体（烟头）检测准确率。此外，运用了一些模型训练的技巧，提升了模型的鲁棒性。由于缺乏现有数据集，自制了一个与吸烟行为相关的数据集。对比实验证明了提出的算法在本数据集以及一些公开数据集上有着更好的检测效果。

关键词: 计算机视觉, 微型目标检测, 实时性, 吸烟检测, 鲁棒性

CHEN Ruilong, LUO Lei, CAI Zhiping, MA Wentao. Algorithm for Real-Time Smoking Detection Based on Deep Learning[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(2): 327-337.

陈睿龙, 罗磊, 蔡志平, 马文涛. 基于深度学习的实时吸烟检测算法[J]. 计算机科学与探索, 2021, 15(2): 327-337.

References

[1] SENYUREK V, IMTIAZ M, BELSARE P, et al. Cigarette smoking detection with an inertial sensor and a smart lighter[J]. Sensors, 2019, 19(3): 570-588.
[2] SENYUREK V Y, IMTIAZ M H, BELSARE P, et al. Smoking detection based on regularity analysis of hand to mouth ges-tures[J]. Biomedical Signal Processing and Control, 2019, 51: 106-112.
[3] WU P, HEIEH J W, CHENG J C, et al. Human smoking event detection using visual interaction clues[C]//Proceedings of the 20th International Conference on Pattern Recognition, Istanbul, Aug 23-26, 2010. Washington: IEEE Computer Society, 2010: 4344-4347.
[4] KRIZHEVSKY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[C]//Proceedings of the 25th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2012: 1097-1105.
[5] REDMON J, DIVVALA S, GIRSHICK R, et al. You only look once: unified, real-time object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 779-788.
[6] VIOLA P A, JONES M J. Rapid object detection using a boosted cascade of simple features[C]//Proceedings of the 2001 IEEE Conference on Computer Vision and Pattern Recogni-tion, Kauai, Dec 8-14, 2001. Washington: IEEE Computer Society, 2001: 1-1.
[7] DALAL N, TRIGGS B. Histograms of oriented gradients for human detection[C]//Proceedings of the 2005 IEEE Conference on Computer Vision and Pattern Recognition, San Diego, Jun 20-25, 2005. Washington: IEEE Computer Society, 2005: 886-893.
[8] FELZENSZWALB P, MCALLESTER D, RAMANAN D. A discriminatively trained, multiscale, deformable part model[C]//Proceedings of the 2008 IEEE Conference on Computer Vision and Pattern Recognition, Anchorage, Jun 23-28, 2008. Washington: IEEE Computer Society, 2008: 1-8.
[9] SIMONYAN K, ZISSERMAN A. Very deep convolutional networks for large-scale image recognition[J]. arXiv:1409.1556, 2014.
[10] SZEGEDY C, VANHOUCKE V, IOFFE S, et al. Rethinking the inception architecture for computer vision[C]//Proceed-ings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 2818-2826.
[11] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Con-ference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[12] REN S, HE K, GIRSHICK R, et al. Faster R-CNN: towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelli-gence, 2015, 39(6): 1137-1149.
[13] LIN T Y, DOLLáR P, GIRSHICK R B, et al. Feature pyramid networks for object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 936-944.
[14] DAI J F, LI Y, HE K M, et al. R-FCN: object detection via region-based fully convolutional networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems, Red Hook, May 20, 2016. Red Hook: Curran Associates, 2016: 379-387.
[15] CAI Z, VASCONCELOS N. Cascade R-CNN: delving into high quality object detection[J]. arXiv:1712.00726, 2017.
[16] NEUBECK A, GOOL L J V. Efficient non-maximum suppr-ession[C]//Proceedings of the 18th International Conference on Pattern Recognition, Hong Kong, China, Aug 20-24, 2006. Washington: IEEE Computer Society, 2006: 850-855.
[17] LIU W, ANGUELOV D, ERHAN D, et al. SSD: single shot multibox detector[C]//LNCS 9905: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 8-16, 2016. Heidelberg: Springer, 2016: 21-37.
[18] LIN T Y, GOYAL P, GIRSHICK R, et al. Focal loss for dense object detection[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2999-3007.
[19] TAN M X, PANG R M, LE Q V. EfficientDet: scalable and efficient object detection[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Seattle, 2020. Washington: IEEE Computer Society, 2020: 10778-10787.
[20] HU J, SHEN L, ALBANIE S, et al. Squeeze-and-excitation networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2020, 42(8): 2011-2023.
[21] WOO S, PARK J C, LEE J Y, et al. CBAM: convolutional block attention module[C]//LNCS 11211: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Heidelberg: Springer, 2018: 3-19.
[22] XIE S N, GIRSHICK R, DOLLAR P, et al. Aggregated resi-dual transformations for deep neural networks[C]//Proceed-ings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 1492-1500.
[23] HE K, ZHANG X, REN S, et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2014, 37(9): 1904-1916.
[24] MISRA D. Mish: a self regularized non-monotonic neural activation function[J]. arXiv:1908.08681, 2019.
[25] ZHENG Z H, WANG P, LIU W, et al. Distance-IoU loss: faster and better learning for bounding box regression[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Symposium on Educa-tional Advances in Artificial Intelligence, New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 12993-13000.
[26] YUN S, HAN D, CHUN S, et al. CutMix: regularization strategy to train strong classifiers with localizable features[C]//Proceedings of the 2019 IEEE International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Washington: IEEE Computer Society, 2019: 6023-6032.