Journal of Frontiers of Computer Science and Technology ›› 2020, Vol. 14 ›› Issue (1): 159-170.DOI: 10.3778/j.issn.1673-9418.1905013

Previous Articles     Next Articles

Multi-Scale Generative Adversarial Networks Image Inpainting Algorithm

LI Kewen, ZHANG Wentao, SHAO Mingwen, LI Le   

  1. College of Computer and Communication Engineering, China University of Petroleum, Qingdao, Shandong 266000, China
  • Online:2020-01-01 Published:2020-01-09

多尺度生成式对抗网络图像修复算法

李克文张文韬邵明文李乐   

  1. 中国石油大学(华东) 计算机与通信工程学院,山东 青岛 266000

Abstract: As a research hotspot in the field of deep learning, image inpainting has important significance in people??s real life. Existing image inpainting methods have various problems that cause visual failure to meet people??s requirements. Aiming at the defects of low accuracy, poor visual consistency and unstable training, this paper proposes an image inpainting algorithm based on the generative adversarial network (GAN) model. The algorithm mainly improves the network structure of the discriminator, and introduces a multi-scale discriminator based on the global discriminator and the local discriminator. The multi-scale discriminators are trained on images of different resolutions. Different scale discriminators have different receptive fields, and guide the generator to generate a more global image view and finer details. Aiming at the problem of gradient disappearance or gradient explosion that often occurs in GAN training, the idea of Wasserstein GAN (WGAN) is adopted, and the EM distance is used to simulate the sample data distribution. The network model of the algorithm is trained and tested on the CelebA, ImageNet and Place2 image datasets. The results show that compared with the previous algorithm models, this algorithm improves the accuracy of image inpainting, can generate more realistic inpainting images, and is suitable for many types of image inpainting.

Key words: image inpainting, generative adversarial network, multi-scale, reconstruction loss, adversarial loss

摘要: As a research hotspot in the field of deep learning, image inpainting has important significance in people??s real life. Existing image inpainting methods have various problems that cause visual failure to meet people??s requirements. Aiming at the defects of low accuracy, poor visual consistency and unstable training, this paper proposes an image inpainting algorithm based on the generative adversarial network (GAN) model. The algorithm mainly improves the network structure of the discriminator, and introduces a multi-scale discriminator based on the global discriminator and the local discriminator. The multi-scale discriminators are trained on images of different resolutions. Different scale discriminators have different receptive fields, and guide the generator to generate a more global image view and finer details. Aiming at the problem of gradient disappearance or gradient explosion that often occurs in GAN training, the idea of Wasserstein GAN (WGAN) is adopted, and the EM distance is used to simulate the sample data distribution. The network model of the algorithm is trained and tested on the CelebA, ImageNet and Place2 image datasets. The results show that compared with the previous algorithm models, this algorithm improves the accuracy of image inpainting, can generate more realistic inpainting images, and is suitable for many types of image inpainting.

关键词: image inpainting, generative adversarial network, multi-scale, reconstruction loss, adversarial loss