Review of Presentation Attack Detection in Face Recognition System

doi:10.3778/j.issn.1673-9418.2012010

Abstract

Abstract:

As the rapid development of face recognition technologies, face presentation attack detection (face liveness detection) techniques are facing new requirements, including instantaneity of?detection, generalization in complicated environments, robustness against various attack types, friendliness of user experience and etc. In?this?paper, the necessity of face presentation attack detection is explained, and the methods are classified, compared and summarized. In general, they can be divided as methods based on manual features and methods based on deep learning. Furthermore, the recent researches on algorithm generalization are summarized as the following categories: methods based on auxiliary supervision, methods based on domain adaptive or domain generalization, methods based on disentangled representation, methods based on noise modeling, and methods based on anomaly detection. Representative algorithm of each type of method are analyzed, and the basic ideas of each method are summarized in detail, as well as the merit and demerit. Issues surrounding face presentation attack detections are systematically summarized from different perspectives, including different types of presentation attacks, advanced detection methods, popular public databases, the standardized evaluation indicators, and testing method. In addition, the difficulties and challenges in this field are also discussed, and the future research direction and development trend are summarized.

Key words: face recognition, presentation attack detection, deep learning, texture?features

摘要：

人脸识别系统的快速发展对人脸活体检测技术提出了新要求，包括检测实时性、面对复杂环境的泛化性、对多种攻击类型的鲁棒性以及用户体验的友好性等。主要阐述了人脸活体检测的必要性，对方法进行了分类、整理和总结，根据所提特征的不同，将活体检测分为基于手工特征的方法和基于深度学习的方法，并将近期针对算法泛化性的研究进展归纳为基于辅助监督信号方法、基于域适应域泛化的方法、基于特征解耦的方法、基于噪声建模的方法、基于异常检测的方法，对每类方法的代表性算法进行了分析介绍，详细总结了每类方法的基本思想和优缺点。从各方面系统地概括了人脸活体检测问题，包括不同类型的呈现攻击、先进的人脸活体检测方法、常用公共数据库、标准化评价指标、测试方法等的介绍。此外，还讨论了该领域的难点与挑战，总结了未来的研究方向和发展趋势。

关键词: 人脸识别, 活体检测, 深度学习, 纹理特征

MA Yukun, XU Yaowen, ZHAO Xin, XU Tao, WANG Zerui. Review of Presentation Attack Detection in Face Recognition System[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(7): 1195-1206.

马玉琨, 徐姚文, 赵欣, 徐涛, 王泽瑞. 人脸识别系统的活体检测综述[J]. 计算机科学与探索, 2021, 15(7): 1195-1206.

References

[1] YAN Y, ZHANG Y J. State-of-the-art on video-based face recognition[J]. Chinese Journal of Computers, 2009, 32(5): 878-886.
严严, 章毓晋. 基于视频的人脸识别研究进展[J]. 计算机学报, 2009, 32(5): 878-886.
[2] MA Y K, HE J Y, WU L F, et al. An effective face verifi-cation algorithm to fuse complete features in convolutional neural network[C]//LNCS 9517: Proceedings of the 22nd International Conference on Multimedia Modeling, Miami, Jan 4-6, 2016. Berlin, Heidelberg: Springer, 2016: 39-46.
[3] GAN J Y, ZHAI Y K, XIANG L, et al. Spatial-temporal tex-ture cascaded feature method for face liveness detection[J]. Pattern Recognition and Artificial Intelligence, 2019, 32(2): 117-123.
甘俊英, 翟懿奎, 项俐, 等. 面向活体人脸检测的时空纹理特征级联方法[J]. 模式识别与人工智能, 2019, 32(2): 117-123.
[4] XIE Z H, JIANG P, YU X H, et al. Hyperspectral face rec-ognition system based on VGGNet and multi-band recur-rent network[J]. Journal of Computer Applications, 2019, 39(2): 388-391.
谢志华, 江鹏, 余新河, 等. 基于VGGNet和多谱带循环网络的高光谱人脸识别系统[J]. 计算机应用, 2019, 39(2): 388-391.
[5] MA Y K, WU L F, GU X F, et al. A secure face-verification scheme based on homomorphic encryption and deep neural networks[J]. IEEE Access, 2017, 5: 16532-16538.
[6] RAMACHANDRA R, BUSCH C. Presentation attack dete-ction methods for face recognition systems: a comprehen-sive survey[J]. ACM Computing Surveys, 2017, 50(1): 8.
[7] SUN P C, ZENG D, LI X Y, et al. A 3D mask presentation attack detection method based on polarization medium wave infrared imaging[J]. Symmetry, 2020, 12(3): 376.
[8] WU L F, MA Y K, ZHOU P, et al. Review of biometric tem-plate protection[J]. Chinese Journal of Scientific Instrument, 2016, 37(11): 2407-2420.
毋立芳, 马玉琨, 周鹏, 等. 生物特征模板保护综述[J]. 仪器仪表学报, 2016, 37(11): 2407-2420.
[9] SAJJAD M, KHAN S, HUSSAIN T, et al. CNN-based anti-spoofing two-tier multi-factor authentication system[J]. Pat-tern Recognition Letters, 2019, 126: 123-131.
[10] TOLOSANA R, GOMEZ-BARRERO M, BUSCH C, et al. Biometric presentation attack detection: beyond the visible spectrum[J]. IEEE Transactions on Information Forensics and Security, 2019, 15: 1261-1275.
[11] LI L, XIA Z Q, HADID A, et al. Replayed video attack dete-ction based on motion blur analysis[J]. IEEE Transactions on Information Forensics and Security, 2019, 14(9): 2246-2261.
[12] AKHTAR Z, MICHELONI C, FORESTI G L. Biometric liveness detection: challenges and research opportunities[J]. IEEE Security & Privacy, 2015, 13(5): 63-72.
[13] BHATTACHARJEE S, MOHAMMADI A, MARCEL S. Spoofing deep face recognition with custom silicone masks [C]//Proceedings of the 9th IEEE International Conference on Biometrics Theory, Applications and Systems, Redondo Beach, Oct 22-25, 2018. Piscataway: IEEE, 2018: 1-7.
[14] HERNANDEZ-ORTEGA J, FIERREZ J, MORALES A, et al. Introduction to face presentation attack detection[M]// Handbook of Biometric Anti-Spoofing. Berlin, Heidelberg: Springer, 2019.
[15] LIU A, WAN J, ESCALERA S, et al. Multi-modal face anti-spoofing attack detection challenge at CVPR2019[C]//Pro-ceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Pis-cataway: IEEE, 2019: 1601-1610.
[16] KOMULAINEN J, BOULKENAFET Z, AKHTAR Z. Review of face presentation attack detection competitions[M]//Hand-book of Biometric Anti-Spoofing. Berlin, Heidelberg: Spr-inger, 2019.
[17] ANJOS A, KOMULAINEN J, MARCEL S, et al. Face anti-spoofing: visual approach[M]//Handbook of Biometric Anti- Spoofing. Berlin, Heidelberg: Springer, 2014.
[18] GALBALLY J, MARCEL S, FIERREZ J. Biometric anti-spoofing methods: a survey in face recognition[J]. IEEE Access, 2014, 2: 1530-1552.
[19] ZHANG J, ZHANG N N. Research on interactive face de-tection based on optimized feature extraction[J]. Computer Engineering and Applications, 2019, 55(13): 193-200.
张进, 张娜娜. 优化特征提取的互动式人脸活体检测研究[J]. 计算机工程与应用, 2019, 55(13): 193-200.
[20] KOLLREIDER K, FRONTHALER H, FARAJ M I, et al. Real-time face detection and motion analysis with applica-tion in liveness assessment[J]. IEEE Transactions on Infor-mation Forensics and Security, 2007, 2(3): 548-558.
[21] ZHOU Y Q, SHI B E. Photorealistic facial expression synth-esis by the conditional difference adversarial autoencoder [C]//Proceedings of the 2017 IEEE International Conference on Affective Computing and Intelligent Interaction, San An-tonio, Oct 23-26, 2017. Washington: IEEE Computer Society, 2017: 370-376.
[22] LIU Z L, SONG G X, CAI J F, et al. Conditional adversarial synthesis of 3D facial action units[J]. Neurocomputing, 2019, 355: 200-208.
[23] KIM S, BAN Y, LEE S. Face liveness detection using a light field camera[J]. Sensors, 2014, 14(12): 22471-22499.
[24] XIE X H, GAO Y, ZHENG W S, et al. One-snapshot face antispoofing using a light field camera[C]//LNCS 10568: Pro-ceedings of the Chinese Conference on Biometric Recogni-tion, Shenzhen, Oct 27-29, 2017. Berlin, Heidelberg: Spr-inger, 2017: 108-117.
[25] LIN Q, LI W J, NING X, et al. Liveness detection using tex-ture and 3D structure analysis[C]//LNCS 9967: Proceedings of the Chinese Conference on Biometric Recognition, Cheng-du, Oct 14-16, 2016. Berlin, Heidelberg: Springer, 2016: 637-645.
[26] SUN X D, HUANG L, LIU C P. Context based face spoof-ing detection using active near-infrared images[C]//Proceed-ings of the 23rd International Conference on Pattern Recogni-tion, Cancún, Dec 4-8, 2016. Piscataway: IEEE, 2016: 4262-4267.
[27] HERNANDEZ-ORTEGA J, FIéRREZ J, MORALES A, et al. Time analysis of pulse-based face anti-spoofing in visible and NIR[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-22, 2018. Piscataway: IEEE, 2018: 544-552.
[28] GEORGE A, MOSTAANI Z, GEISSENBUHLER D, et al. Biometric face presentation attack detection with multi-channel convolutional neural network[J]. IEEE Transactions on Information Forensics and Security, 2020, 15: 42-55.
[29] ZHANG S, WANG X, LIU A, et al. CAISIA-SURF: a data-set and benchmark for large-scale multi-modal face anti-spoofing[J]. IEEE Transactions on Biometrics, Behavior, and Identity Science, 2020, 2(2): 182-193.
[30] MAATTA J, HADID A, PIETIKAINEN M. Face spoofing detection from single images using texture and local shape analysis[J]. IET Biometrics, 2012, 1(1): 3-10.
[31] DI W, HU H, JAIN A K. Face spoof detection with image distortion analysis[J]. IEEE Transactions on Information Forensics and Security, 2015, 10(4): 746-761.
[32] LI J W, WANG Y H, TAN T N, et al. Live face detection based on the analysis of Fourier spectra[C]//Proceedings of SPIE 5404, Biometric Technology for Human Identification, Orlando, Apr 12-13, 2004. International Society for Optics and Photonics, 2004: 296-304.
[33] GARCIA D C, DE QUEIROZ R L. Face-spoofing 2D-detection based on Moiré-pattern analysis[J]. IEEE Transac-tions on Information Forensics and Security, 2015, 10(4):778-786.
[34] PATEL K, HAN H, JAIN A K, et al. Live face video vs. spoof face video: use of Moiré patterns to detect replay video att-acks[C]//Proceedings of the 2015 IEEE International Confer-ence on Biometrics, Phuket, May 19-22, 2015. Piscataway: IEEE, 2015: 98-105.
[35] PAN G, SUN L, WU Z H, et al. Eyeblink-based anti-spoofing in face recognition from a generic webcamera[C]//Proceed-ings of the IEEE 11th International Conference on Computer Vision, Rio de Janeiro, Oct 14-20, 2007. Washington: IEEE Computer Society, 2007: 1-8.
[36] GIBSON J J. The perception of the visual world[M]. Boston: Houghton Mifflin Harcourt, 1950.
[37] KOLLREIDER K, FRONTHALER H, BIGüN J. Evaluat-ing liveness by face images and the structure tensor[C]// Proceedings of the 4th IEEE Workshop on Automatic Iden-tification Advanced Technologies, Buffalo, Oct 16-18, 2005. Washington: IEEE Computer Society, 2005: 75-80.
[38] YANG J, LEI Z, LI S Z. Learn convolutional neural network for face anti-spoofing[J]. arXiv:1408.5601, 2014.
[39] LUCENA O, JUNIOR A, MOIA V, et al. Transfer learning using convolutional neural networks for face anti-spoofing [C]//LNCS 10317: Proceedings of the 14th International Con-ference on Image Analysis and Recognition, Montreal, Jul 5-7, 2017. Berlin, Heidelberg: Springer, 2017: 27-34.
[40] ALOTAIBI A, MAHMOOD A. Enhancing computer vision to detect face spoofing attack utilizing a single frame from a replay video attack using deep learning[C]//Proceedings of the 2016 IEEE International Conference on Optoelectronics and Image Processing, Warsaw, Jun 10-12, 2016. Piscataway: IEEE, 2016: 1-5.
[41] DE SOUZA G B, PAPA J P, MARANA A N. On the learn-ing of deep local features for robust face spoofing detection [C]//Proceedings of the 31st SIBGRAPI Conference on Gra-phics, Patterns and Images, Paraná, Oct 29-Nov 1, 2018. Washington: IEEE Computer Society, 2018: 258-265.
[42] WANG G Q, HAN H, SHAN S G, et al. Cross-domain face presentation attack detection via multi-domain disentangled representation learning[C]//Proceedings of the 2020 IEEE Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 6677-6686.
[43] COSTA-PAZO A, JIMéNEZ-CABELLO D, VAZQUEZ-FERNANDEZ E, et al. Generalized presentation attack det-ection: a face anti-spoofing evaluation proposal[J]. arXiv:1904.06213, 2019.
[44] QIN Y, ZHAO C, ZHU X, et al. Learning meta model for zero-and few-shot face anti-spoofing[J]. arXiv:1904.12490, 2019.
[45] ATOUM Y, LIU Y J, JOURABLOO A, et al. Face anti-spoofing using patch and depth-based CNNs[C]//Proceed-ings of the 2017 IEEE International Joint Conference on Biometrics, Denver, Oct 1-4, 2017. Piscataway: IEEE, 2017: 319-328.
[46] LI X B, CHEN J, ZHAO G Y, et al. Remote heart rate mea-surement from face videos under realistic situations[C]//Pro-ceedings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 23-28, 2014. Pisca-taway: IEEE, 2014: 4264-4271.
[47] LI X B, KOMULAINEN J, ZHAO G Y, et al. Generalized face anti-spoofing by detecting pulse from face videos[C]// Proceedings of the 23rd International Conference on Pattern Recognition, Cancún, Dec 4-8, 2016. Piscataway: IEEE, 2016: 4244-4249.
[48] LAKSHMINARAYANA N N, NARAYAN N, NAPP N, et al. A discriminative spatio-temporal mapping of face for liveness detection[C]//Proceedings of the 2017 IEEE Inter-national Conference on Identity, Security and Behavior An-alysis, New Delhi, Feb 22-24, 2017. Piscataway: IEEE, 2017: 1-7.
[49] LIU Y J, JOURABLOO A, LIU X M. Learning deep models for face anti-spoofing: binary or auxiliary supervision[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Piscataway: IEEE, 2018: 389-398.
[50] LIU S Q, LAN X, YUEN P C. Remote photoplethysmogra-phy correspondence feature for 3D mask face presentation attack detection[C]//LNCS 11220: Proceedings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 558-573.
[51] ZHU X, LI S, ZHANG X, et al. Detection of spoofing me-dium contours for face anti-spoofing[J]. IEEE Transactions on Circuits and Systems for Video Technology, 2019: 1-10.
[52] SHAO R, LAN X, LI J, et al. Multi-adversarial discrimina-tive deep domain generalization for face presentation attack detection[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 10023-10031.
[53] JIA Y, ZHANG J, SHAN S, et al. Single-side domain gen-eralization for face anti-spoofing[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Re-cognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 8484-8493.
[54] SAHA S, XU W, KANAKIS M, et al. Domain agnostic fea-ture learning for image and video based face anti-spoofing[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 14-19, 2020. Piscataway: IEEE, 2020: 3490-3499.
[55] WANG G, HAN H, SHAN S, et al. Unsupervised advers-arial domain adaptation for cross-domain face presentation attack detection[J]. IEEE Transactions on Information For-ensics and Security, 2021, 16: 56-69.
[56] WANG G Q, HAN H, SHAN S G, et al. Improving cross-database face presentation attack detection via adversarial domain adaptation[C]//Proceedings of the 2019 International Conference on Biometrics, Crete, Jun 4-7, 2019. Piscataway: IEEE, 2019: 1-8.
[57] ZHANG K Y, YAO T P, ZHANG J, et al. Face anti-spoofing via disentangled representation learning[C]//LNCS 12364: Proceedings of the 16th European Conference on Computer Vision, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 641-657.
[58] JOURABLOO A, LIU Y J, LIU X M. Face de-spoofing: anti-spoofing via noise modeling[C]//LNCS 11217: Proceedings of the 15th European Conference on Computer Vision, Mu-nich, Sep 8-14, 2018. Cham: Springer, 2018: 297-315.
[59] LIU Y J, STEHOUWER J, LIU X M. On disentangling spoof trace for generic face anti-spoofing[C]//LNCS 12363: Pro-ceedings of the 16th European Conference on Computer Vi-sion, Glasgow, Aug 23-28, 2020. Cham: Springer, 2020: 406-422.
[60] NIKISINS O, MOHAMMADI A, ANJOS A, et al. On eff-ectiveness of anomaly detection approaches against unseen presentation attacks in face anti-spoofing[C]//Proceedings of the 2018 International Conference on Biometrics, Gold Coast, Feb 20-23, 2018. Piscataway: IEEE, 2018: 75-81.
[61] PéREZ-CABO D, JIMéNEZ-CABELLO D, COSTA-PAZO A, et al. Deep anomaly detection for generalized face anti-spoofing[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 1591-1600.
[62] LIU Y J, STEHOUWER J, JOURABLOO A, et al. Deep tree learning for zero-shot face anti-spoofing[C]//Proceeding of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 4680-4689.
[63] BAWEJA Y, OZA P, PERERA P, et al. Anomaly detection-based unknown face presentation attack detection[C]//Pro-ceedings of the 2020 International Joint Conference on Bio-metrics, Houston, Sep 28-Oct 1, 2020. Piscataway: IEEE, 2020: 1-9.
[64] LI A, TAN Z, LI X, et al. CASIA-SURF CeFA: a benchmark for multi-modal cross-ethnicity face anti-spoofing[J]. arXiv: 2003.05136, 2020.
[65] LI H, LI W, CAO H, et al. Unsupervised domain adaptation for face anti-spoo?ng[J]. IEEE Transactions on Information Forensics and Security, 2018, 13(7): 1794-1809.
[66] YANG X, LUO W H, BAO L C, et al. Face anti-spoofing: model matters, so does data[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recogni-tion, Long Beach, Jun 15-20, 2019. Piscataway: IEEE, 2019: 3507-3516.
[67] BANSAL A, MA S, RAMANAN D, et al. Recycle-GAN: unsupervised video retargeting[C]//LNCS 11209: Proceed-ings of the 15th European Conference on Computer Vision, Munich, Sep 8-14, 2018. Cham: Springer, 2018: 122-138.