Journal of Frontiers of Computer Science and Technology ›› 2021, Vol. 15 ›› Issue (9): 1728-1739.DOI: 10.3778/j.issn.1673-9418.2006075

• Graphics and Image • Previous Articles     Next Articles

Aesthetic Feature Image Generation Method Embedded with Self-Attention Mechanism

MA Li, ZOU Yali   

  1. School of Computer, Xi??an University of Posts and Telecommunications, Xi??an 710061, China
  • Online:2021-09-01 Published:2021-09-06

嵌入自注意力机制的美学特征图像生成方法

马力邹亚莉   

  1. 西安邮电大学 计算机学院,西安 710061

Abstract:

In order to solve the problems such as the single structure of the generated image, the lack of rich detail features, resulting in the lack of aesthetic sense, this paper proposes an aesthetic feature image generation method embedded with self-attention mechanism. In order to increase the aesthetic characteristics of the generated image, the correlation between the image aesthetic evaluation criteria and the generation model is studied, and the aesthetic loss function based on aesthetic score is defined. In order to ensure the consistency of the generated image and the real image in semantic content, VGG network is added to construct the content loss function, the Charbonnier loss is used to replace the L1 loss, and the aesthetic loss, content loss and the confrontation loss of the evolutionary generation confrontation network are combined in a weighted form, to guide and optimize image generation. The self-attention mechanism module is introduced into the generator and discriminator, and the dense blocks are added before the self-attention mechanism module of the generator to fully extract the features, which is conducive to the self-attention mechanism to obtain more internal global dependencies of the features, and promoting the generated image to have clear details and rich texture features. The experimental results on Cifar10 and CUHKPQ data sets show that the method is effective in improving the image aesthetic effect. The Fréchet  distance value is increased by 3.21 and 5.44 respectively compared with that of the evolutionary generation confrontation network, and the image aesthetic score value is improved by 0.75 and 0.88 compared with the evolutionary generation confrontation network.

Key words: evolutionary generative confrontation network, image aesthetics, self-attention, loss function, image generation

摘要:

针对生成的图像结构单一,细节特征不够丰富,导致美观感不足等问题,提出了一种嵌入自注意力机制的美学特征图像生成方法。为了增加生成图像的美学特征,研究图像美学评价标准与生成模型之间的关联性,定义了基于美学分数的美学损失函数;为保证生成图像与真实图像在语义内容上的一致性,加入VGG网络,构造内容损失函数,采用Charbonnier损失代替L1损失,并将美学损失、内容损失和进化生成对抗网络的对抗损失以加权形式组合,引导与优化图像的生成。在生成器和判别器中引入自注意力机制模块,并将密集卷积块加入生成器自注意力机制模块之前,充分提取特征,有利于自注意力机制高效获取更多特征内部的全局依赖关系,促使生成图像细节清晰,纹理特征丰富。在Cifar10、CUHKPQ两个数据集上的实验结果表明该方法在提升图像美学效果方面是有效的,其弗雷歇距离值相较于进化生成对抗网络分别提高了3.21和5.44,图像美学分数值相较于进化生成对抗网络分别提高了0.75和0.88。

关键词: 进化生成对抗网络, 图像美学, 自注意力, 损失函数, 图像生成