Journal of Frontiers of Computer Science and Technology ›› 2021, Vol. 15 ›› Issue (10): 1773-1794.DOI: 10.3778/j.issn.1673-9418.2103092

• Surveys and Frontiers • Previous Articles     Next Articles

Review of New Face Occlusion Inpainting Technology Research

LIU Ying, ZHANG Yixuan, SHE Jianchu, WANG Fuping, LIM Kengpang   

  1. 1. Center for Image and Information Processing, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
    2. Key Laboratory of Electronic Information Application Technology for Crime Scene Investigation, Ministry of Public Security, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
  • Online:2021-10-01 Published:2021-09-30

人脸去遮挡新技术研究综述

刘颖张艺轩佘建初王富平林庆帆   

  1. 1. 西安邮电大学 图像与信息处理研究所,西安 710121
    2. 西安邮电大学 电子信息现场勘验应用技术公安部重点实验室,西安 710121

Abstract:

In criminal investigation, if the face image of a suspect is occluded and the face feature points are corrupted, the precise removal of the occluded area becomes an important step in improving face recognition technology. Therefore, face occlusion inpainting has important research significance. The recent progress of face occlusion inpainting technology is described, and based on the deep learning-based image restoration algorithm first proposed in 2016, various face occlusion inpainting fusion algorithms proposed by scholars from 2017 to the present are introduced. Firstly, the existing algorithms are classified into random occlusion and regular occlusion face restoration according to the different ways of occlusion, then further classified into based on convolutional neural network (CNN) and based on generative adversarial network (GAN) according to the different predictive generation networks in the algorithms. This paper analyzes various types of fusion algorithms in terms of model network characteristics, advantages and disadvantages as well as applicable scenarios, and gives some suggestions for the selection of fusion algorithms. The regular occlusion algorithms and random occlusion algorithms are compared and summarized in terms of network structure and applicability range. Then it introduces and summarizes the commonly used image restoration effect evaluation indices and datasets, and by listing the experimental results of various restoration algorithms, it refines and analyzes their quantitative indices and visual effects, and illustrates that the face occlusion inpainting technology has made great progress in recent years. Finally, the future development trend of face occlusion inpainting technology is pointed out from five aspects, such as dataset, algorithm and evaluation index, by combining the existing algorithms and practical requirements.

Key words: face occlusion inpainting, deep learning, convolutional neural network (CNN), generative adversarial network (GAN)

摘要:

刑侦工作中,若犯罪嫌疑人的人脸图像存在遮挡,人脸特征点遭到破坏,精确去除遮挡区域成为提高人脸识别技术的重要一步。因此,人脸去遮挡有着重要的研究意义。对人脸去遮挡技术最新进展进行阐述,并基于2016年首次提出的基于深度学习图像修复算法,介绍从2017年至今学者们提出的各类人脸去遮挡融合算法。首先根据遮挡方式的不同将现有算法分类为随机遮挡和规则遮挡的人脸修复,接着根据算法中预测生成网络的不同,进一步分为基于卷积神经网络(CNN)和基于生成式对抗网络(GAN),并对各类融合算法从模型网络特点、优缺点以及适用场景进行分析,给出一些融合算法的选择建议,从网络结构和适用范围方面对比总结规则遮挡算法和随机遮挡算法。然后介绍并汇总常用的图像修复效果评价指标和数据集,通过列举各类修复算法的实验结果,提炼并分析其定量指标和视觉效果,说明了近年来的人脸去遮挡技术取得了较大的进展。最后结合现有算法和实际需求,从数据集、算法、评价指标等五方面指出人脸去遮挡技术的未来发展趋势。

关键词: 人脸去遮挡, 深度学习;卷积神经网络(CNN);生成式对抗网络(GAN)