关键词: 图像修复, 注意力机制, Transformer, 特征提取

Abstract: Aiming at the problems of incoherent semantic connection between image pixels and ineffective restoration effect of local texture details in large-scale damaged images, this paper proposes a single-stage image restoration network model named BMTA (block of multi-transformer attention). It can be used to repair a large area of damaged images in multiple scenes, so that the repaired images can have a good performance in the subjective perception of human eyes and objective evaluation indicators. The generator module performs feature compression, reconstruction and enhancement of important feature information of the input image by interspersing dual unidirectional attention modules in the convolution layer. The compressed feature information is divided into channels for local feature extraction and global feature extraction. The global information connection is established by using the segmented fringe window, and the local detail information is extracted with depth by using residual dense blocks. The extracted features are fused. In the decoder part, in order to prevent the local information loss caused by the decoding process and the inaccurate understanding of the context information during the restoration process, the gated linear self-attention module is used to ensure the multi-level retention of information in the network, so as to achieve the restoration effect closer to the original image. Finally, a discriminator is used to evaluate the repair results and promote better expressiveness of the repaired images in terms of structure and texture. The proposed method in this paper performs better than the current advanced image restoration algorithms on CelebA, StreetView, and Places2 datasets.

Key words: image inpainting, attention mechanism, Transformer, feature extraction