Journal of Frontiers of Computer Science and Technology ›› 2021, Vol. 15 ›› Issue (1): 150-162.DOI: 10.3778/j.issn.1673-9418.2001035

• Graphics and Image • Previous Articles     Next Articles

Face Inpainting Algorithm Combining Edge Information with Gated Convolution

WANG Fuping, LI Wenlou, LIU Ying, LU Jin, GONG Yanchao   

  1. 1. Key Laboratory of Electronic Information Application Technology for Crime Scene Investigation, Ministry of Public Security, Xi'an University of Posts and Telecommunications, Xi'an 710121, China
    2. Center for Image and Information Processing, Xi'an University of Posts and Telecommunications, Xi'an 710121, China
    3. International Joint Research Center for Wireless Communication and Information Processing Technology, Shaanxi Province, Xi'an 710121, China
  • Online:2021-01-01 Published:2021-01-07

结合边缘信息和门卷积的人脸修复算法

王富平李文楼刘颖卢津公衍超   

  1. 1. 西安邮电大学 电子信息现场勘验应用技术公安部重点实验室,西安 710121
    2. 西安邮电大学 图像与信息处理研究所,西安 710121
    3. 陕西省无线通信与信息处理技术国际合作研究中心,西安 710121

Abstract:

For the face inpainting under arbitrary shape occlusion, the existing methods are easy to produce edge blur and distortion of the inpainting results. In this paper, an algorithm for face inpainting combining edge information with gated convolution is proposed. Firstly, the edge image of occluded area is generated by prior face knowledge to constrain the process of face inpainting. Secondly, the gated convolution holds the ability to extract accurate local feature in the absence of some pixels, and a gated convolution-based generative adversarial network (GAN) for image inpainting is designed. The model consists of two parts: edge connection GAN and image inpainting GAN. The edge connection network uses the binary occlusion image, the image to be repaired and its edge image for training, and realizes the automatic completion and connection of the missing edge image. The image inpainting GAN takes the completed edge image as the guidance information, and combines the occlusion image to repair the missing area. The experimental results show that the inpainting effect of this algorithm is better than that of other algorithms, and its evaluation indicators are better than those of the current image inpainting algorithms based on deep learning.

Key words: face inpainting, Canny edge, gated convolution, deep learning, generative adversarial network (GAN)

摘要:

针对任意形状遮挡下人脸修复,现有方法容易产生边缘模糊和恢复结果失真等问题。提出了一种结合边缘信息和门卷积的人脸修复算法。首先,通过先验人脸知识产生遮挡区域的边缘图,以约束人脸修复过程。其次,利用门卷积在部分像素缺失下的精确局部特征描述能力,设计面向图像修复的门卷积深度生成对抗网络(GAN)。该模型由边缘连接生成对抗网络和图像修复生成对抗网络两部分组成。边缘连接网络利用二值遮挡图和待修复图像及其边缘图的多源信息进行训练,实现对缺失边缘图像的自动补全和连接。图像修复网络以补全的边缘图为引导信息,联合遮挡图像进行缺失区域修复。实验结果表明:相比其他算法,该算法修复效果更好,其评价指标比当前基于深度学习的图像修复算法更优。

关键词: 人脸修复, Canny边缘, 门卷积, 深度学习, 生成对抗网络(GAN)