Abstract: Remote sensing image data have a large volume and a wide range of land cover categories. There is a significant disparity between local and global features, and noticeable differences in the intra-domain features. This makes it challenging for traditional transfer learning to effectively improve the model’s generalization performance. In light of this, based on the traditional unsupervised domain adaptation model that aligns features between domains, a prototype-combined two-stage unsupervised domain adaptation segmentation model for remote sensing images is proposed. Firstly, category features are represented using prototypes. The prototype acquisition module is introduced to obtain and update prototypes. By applying prototypes through the prototype imposition module in combination with self-attention, global category features are imposed on the locally cropped image features. This enables the segmentation network to consider both local and global category information, thus better extracting invariant features from both domains. Secondly, the target domain images are divided into easy and hard segments using pseudo-labels. Through adversarial and self-training methods, the intra-domain feature differences in the target domain are reduced, facilitating better extraction of intra-domain invariant features from easy and hard target domain images. Lastly, the segmentation prediction maps, which contain known pixel categories, are used to compute the contextual relationships between each pixel and its neighboring pixels. A pixel context relationship graph is generated to determine the domain from which the pixel context relationship in the output-level discriminative network originates. This compels the segmentation network to better extract domain-invariant contextual relationships and alleviate the problem of spectral confusion. Experimental results on two datasets demonstrate that the proposed model effectively mitigates the challenges posed by large disparities between local and global features, significant intra-domain feature differences, and the issue of spectral confusion, leading to a decline in model transfer performance. Compared with advanced domain adaptation segmentation methods, the proposed model exhibits superior performance.

Key words: image segmentation, remote sensing images, unsupervised domain adaptation, global and local features, pixel contextual relationship

摘要： 遥感图像数据量较大，地物类别较多，局部特征与全局特征差距较大，域内特征差异较明显，导致传统的迁移学习难以有效提升模型的推广性能。为此，在传统基于对抗对齐域间特征的无监督域适应模型基础上，提出一种结合原型的两阶段遥感图像无监督域适应分割模型。引入原型表达类别特征，通过原型获取模块获取和更新原型，利用原型施加模块并结合自注意力，将类别全局特征施加到裁剪得到的局部图像特征中，使分割网络兼顾局部和全局类别信息，更好地提取两个域的不变特征。利用目标域图像的伪标签将目标域图像分为易分割和难分割图像，通过对抗和自训练的方式减少目标域的域内特征差异，以便更好地提取目标域难易图像的域内不变特征。利用已知像素类别的分割预测图计算每个像素和相邻像素的上下文关系，生成像素上下文关系图，通过使输出级判别网络判别输入的两个域分割结果的像素上下文关系图来自于哪个域，迫使分割网络更好地提取域不变上下文关系，缓解同谱异物现象。在两个数据集上的实验结果表明，所提模型可以有效缓解局部特征与全局特征差距较大、域内特征差异明显以及同谱异物现象带来的模型迁移性能下降问题，相较于先进的域适应分割方法更具有优势。

关键词: 图像分割, 遥感图像, 无监督域适应, 全局和局部特征, 像素上下文关系