用于无监督域适应的深度对抗重构分类网络

doi:10.3778/j.issn.1673-9418.2012067

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (5): 1107-1116.DOI: 10.3778/j.issn.1673-9418.2012067

用于无监督域适应的深度对抗重构分类网络

林佳伟¹^,⁺(), 王士同²

1.江南大学人工智能与计算机学院,江苏无锡 214122
2.江南大学江苏省媒体设计与软件技术重点实验室,江苏无锡 214122

收稿日期:2020-12-03 修回日期:2021-01-27 出版日期:2022-05-01 发布日期:2022-05-19
通讯作者: + E-mail: jnuljw@163.com
作者简介:林佳伟（1996—）,男,山东威海人,硕士研究生,主要研究方向为人工智能、模式识别。
王士同（1964—）,男,江苏扬州人,教授,博士生导师,CCF会员,主要研究方向为人工智能、模式识别等。
基金资助:
江苏省自然科学基金(BK20191331)

Deep Adversarial-Reconstruction-Classification Networks for Unsupervised Domain Adaptation

LIN Jiawei¹^,⁺(), WANG Shitong²

1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China
2. Key Laboratory of Media Design and Software Technology of Jiangsu Province, Jiangnan University, Wuxi, Jiangsu 214122, China

Received:2020-12-03 Revised:2021-01-27 Online:2022-05-01 Published:2022-05-19
About author:LIN Jiawei, born in 1996, M.S. candidate. His research interests include artificial intelligence and pattern recognition.
WANG Shitong, born in 1964, professor, Ph.D. supervisor, member of CCF. His research interests include artificial intelligence, pattern recognition, etc.
Supported by:
Natural Science Foundation of Jiangsu Province(BK20191331)

摘要/Abstract

摘要：

最近迁移学习的新方法对抗域适应,将生成对抗网络（GAN）的思想添加到深度网络中,能够学习数据的可迁移表示形式进行域适应。虽然通过GAN的思想能够很好地提取出源域数据和目标域数据的共同特征,有效地进行不同域之间的知识迁移,但现有的对抗域适应算法不能有效地保留目标域数据的局部特征,而目标域数据的某些特征可能会对分类精度有显著的提升。为了避免原始数据的局部特征因对抗性学习遭到破坏,利用多任务神经网络来保留目标域数据的局部特征。提出了一个深度对抗重构分类网络的模型（DARCN）。DARCN受到自动编码器的启发,在对抗域适应的基础上,添加了自动编码器的解码部分,这样能够有效地从低维特征重建原始数据。该模型学习了以下任务的共享编码表示：带标签的源域数据的监督分类;不带标签的目标域数据的无监督重构;源域和目标域的不可区分性。最后,最小化标签分类器的分类损失和解码器的重构损失,同时最大化域判别器的分类损失,通过梯度下降法能够有效地解决此类优化问题。实验结果证明了目标域局部特征的保留对领域自适应任务是十分关键的。

关键词: 迁移学习, 对抗域适应, 生成对抗网络（GAN）, 深度对抗重构分类网络（DARCN）, 自动编码器

Abstract:

Recently, a new method of transfer learning called adversarial domain adaptation, embeds the idea of the generative adversarial networks (GAN) into the deep networks. It can learn the transferable representation of data for domain adaptation by the thought of the GAN. Although this method can extract the common features of the source domain data and target domain data, and effectively transfer knowledge between different domains, the existing adversarial domain adaptation algorithms cannot effectively retain the local features of the target domain. However, some features of the target domain data may significantly improve the classification accuracy. In order to avoid the destruction of the local features of the original data due to adversarial learning, a multi-task neural network is used to retain the local features of the target domain data. A model of deep adversarial-reconstruction-classification networks (DARCN) is proposed. DARCN is inspired by the auto-encoder. On the basis of adversarial domain adap-tation, the decoding part of the auto-encoder is added, which can effectively reconstruct the original data from low-dimensional features. The model learns shared coding representations for the following tasks: supervised classification of labeled source domain data, unsupervised reconstruction of unlabeled target domain data and indistinguishability of source domain and target domain. Finally, the classification loss of the label classifier and the reconstruction loss of the decoder are minimized, and the classification loss of the domain discriminator is maximized. The gradient descent method can effectively solve such optimization problems. The experimental results prove that the preservation of local features of target domain is critical for domain adaptation tasks.

Key words: transfer learning, adversarial domain adaptation, generative adversarial networks (GAN), deep adversarial-reconstruction-classification networks (DARCN), auto-encoder

中图分类号:

TP181

林佳伟, 王士同. 用于无监督域适应的深度对抗重构分类网络[J]. 计算机科学与探索, 2022, 16(5): 1107-1116.

LIN Jiawei, WANG Shitong. Deep Adversarial-Reconstruction-Classification Networks for Unsupervised Domain Adaptation[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(5): 1107-1116.

图/表 9

图1 DARCN架构图

Fig.1 Architecture of DARCN

图2 实验1中的网络结构图

Fig.2 Network structure diagram in experiment 1

图3 实验2、实验3中的网络结构图

Fig.3 Network structure diagram in experiments 2 and 3

图4 不同的超参数 μ对DARCN模型性能的影响

Fig.4 Influence of different parameter μon DARCN

表1 数字数据集运行10次平均分类精度

Table 1 Average classification accuracy of 10 times on digital datasets %

Methods	MNIST MNIST-M	SVHN MNIST	MNIST USPS	Average
CovNet_src	52.2	54.9	85.5	64.2
SDA	42.6	55.2	43.1	46.7
SA	56.7	59.3	85.9	67.3
DANN	80.3	66.6	88.4	78.4
DARCN	86.4	72.2	94.7	84.4
CovNet_tgt	95.9	99.4	96.1	97.1

图5 源域图像（MNIST）、重构图像、目标域图像（MNIST-M）的对比

Fig.5 Comparison of source domain (MNIST) image, reconstructed image and target domain (MNIST-M) image

图6 基于AlexNet网络的DARCN结构

Fig.6 DARCN structure based on AlexNet

表2 Office-Caltech10数据集运行10次平均分类精度

Table 2 Average classification accuracy of 10 times on Office-Caltech10 dataset %

Methods	WebcamAmazon	Dslr Amazon	Dslr Caltech	Dslr Webcam	WebcamCaltech	Average
CovNet_src	41.2	35.2	35.7	67.2	38.8	43.6
JGSA	40.8	38.7	30.3	93.2	33.6	47.3
DDC	72.1	42.5	43.7	71.0	69.4	59.7
DANN	63.0	62.7	57.9	88.9	55.6	65.6
DARCN	65.8	66.0	60.6	89.8	59.8	68.4
CovNet_tgt	81.3	81.3	70.6	93.3	70.6	79.4

表3 DARCN参数规模分析

Table 3 Analysis of parameter scale in DARCN

网络	输入维度	参数量/10⁴	内存占用/MB	计算量/10⁴
网络1	28×28×3	34	1.30	3 152
网络2	32×32×3	66	2.52	5 680
网络3	224×224×3	21 636	825.35	414 076

参考文献 30

[1]	SIMONYAN K, ZISSERMAN A. Very deep convolutional net-works for large-scale image recognition[J]. arXiv:1409.1556, 2014.
[2]	HE K M, ZHANG X Y, REN S Q, et al. Deep residual lear-ning for image recognition[C]// Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recogni-tion, Las Vegas, Jun 26-Jul 1, 2016. Washington: IEEE Com-puter Society, 2016: 770-778.
[3]	LONG M S, WANG J M, DING G G, et al. Transfer joint matching for unsupervised domain adaptation[C]// Proceed-ings of the 2014 IEEE Conference on Computer Vision and Pattern Recognition, Columbus, Jun 21-23, 2014. Washing-ton: IEEE Computer Society, 2014: 1410-1417.
[4]	ALJUNDI R, EMONET R, MUSELET D, et al. Landmarks-based kernelized subspace alignment for unsupervised domain adaptation[C]// Proceedings of the 2015 IEEE Conference on Computer Vision and Pattern Recognition, Boston, Jun 7-12, 2015. Washington: IEEE Computer Society, 2015: 56-63.
[5]	毛发贵, 李碧雯, 沈备军. 基于实例迁移的跨项目软件缺陷预测[J]. 计算机科学与探索, 2016, 10(1): 43-55.
	MAO F G, LI B W, SHEN B J. Cross-project software de-fect prediction based on instance transfer[J]. Journal of Fron-tiers of Computer Science and Technology, 2016, 10(1): 43-55.
[6]	LONG M S, WANG J M, DING G G, et al. Adaptation regula-rization: a general framework for transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2014, 26(5): 1076-1089. DOI URL
[7]	WANG J D, FENG W J, CHEN Y Q, et al. Visual domain adaptation with manifold embedded distribution alignment[J]. arXiv:1807.07258, 2018.
[8]	SUN B C, FENG J S, SAENKO K. Return of frustratingly easy domain adaptation[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Feb 12-17, 2016. Menlo Park: AAAI, 2016: 2058-2065.
[9]	YAN K, KOU L, ZHANG D. Learning domain-invariant sub-space using domain features and independence maximization[J]. IEEE Transactions on Systems, Man, and Cybernetics, 2018, 48(1): 288-299.
[10]	许鹏, 邓赵红, 王骏, 等. 基于联合信息保持的异构领域自适应[J]. 计算机科学与探索, 2020, 14(7): 1183-1193.
	XU P, DENG Z H, WANG J, et al. Joint information pre-servation for heterogeneous domain adaptation[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(7): 1183-1193.
[11]	TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial dis-criminative domain adaptation[C]// Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recogni-tion, Hawaii, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 7167-7176.
[12]	MOTIIAN S, PICCIRILLI M, ADJEROH D A, et al. Uni-fied deep supervised domain adaptation and generalization[C]// Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 5715-5725.
[13]	刘建伟, 孙正康, 罗雄麟. 域自适应学习研究进展[J]. 自动化学报, 2014, 40(8): 1576-1600.
	LIU J W, SUN Z K, LUO X L. Review and research devel-opment on domain adaptation learning[J]. Acta Automatica Sinica, 2014, 40(8): 1576-1600.
[14]	PRATT L Y. Discriminability-based transfer between neural net-works[C]// Advances in Neural Information Processing Syst-ems 5, Denver, Nov 30-Dec 3, 1992. San Mateo: Morgan Kauf-mann, 1992: 204-211.
[15]	BEN-DAVID S, BLITZER J, CRAMMER K, et al. Analysis of representations for domain adaptation[C]// Proceedings of the 20th Annual Conference on Neural Information Proce-ssing Systems, Vancouver, Dec 4-7, 2006. Cambridge: MIT Press, 2007: 137-144.
[16]	BLITZER J, CRAMMER K, KULESZA A, et al. Learning bounds for domain adaptation[C]// Proceedings of the 21st Annual Conference on Neural Information Processing Sys-tems, Vancouver, Dec 3-6, 2007. Red Hook: Curran Asso-ciates, 2008: 129-136.
[17]	YOSINSKI J, CLUNE J, BENGIO Y, et al. How transferable are features in deep neural networks?[C]// Proceedings of the Annual Conference on Neural Information Processing Systems 2014, Montreal, Dec 8-13, 2014: 3320-3328.
[18]	FERNANDO B, HABRARD A, SEBBAN M, et al. Subspace alignment for domain adaptation[J]. arXiv:1409.5241, 2014.
[19]	GLOROT X, BORDES A, BENGIO Y. Domain adaptation for large-scale sentiment classification: a deep learning approach[C]// Proceedings of the 28th International Conference on Machine Learning, Bellevue, Jun 28-Jul 2, 2011. Madison: Omnipress, 2011: 513-520.
[20]	CHOPRA S, BALAKRISHNAN S, GOPALAN R. Dlid: deep learning for domain adaptation by interpolating bet-ween domains[C]// Proceedings of the 2013 Workshop on Chal-lenges in Representation Learning, Atlanta, 2013: 11690995.
[21]	YOU K C, KOU Z, LONG M S. Co-tuning for transfer learning[C]// Proceedings of the Annual Conference on Neural Information Processing Systems 2020, Dec 6-12, 2020: 1-11.
[22]	GANIN Y, USTINOVA E, AJAKAN H, et al. Domain-adversarial training of neural networks[J]. The Journal of Machine Learning Research, 2016, 17(1): 2096-2030.
[23]	GOODFELLOW I J, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. arXiv:1406.2661, 2014.
[24]	KHAN M Z, JABEEN S, KHAN M U G, et al. A realistic image generation of face from text description using the fully trained generative adversarial networks[J]. IEEE Access, 2021, 9: 1250-1260. DOI URL
[25]	ZHANG F, WANG C. MSGAN: generative adversarial net-works for image seasonal style transfer[J]. IEEE Access, 2020, 8: 104830-104840. DOI URL
[26]	吴春梅, 胡军浩, 尹江华. 利用改进生成对抗网络进行人体姿态识别[J]. 计算机工程与应用, 2020, 56(8): 96-103.
	WU C M, HU J H, YIN J H. Using improved generative adversarial network for human pose estimation[J]. Computer Engineering and Applications, 2020, 56(8): 96-103.
[27]	IENCO D, PENSA R G. Enhancing graph-based semisuper-vised learning via knowledge-aware data embedding[J]. IEEE Transactions on Neural Networks and Learning Systems, 2020, 31(11): 5014-5020. DOI URL
[28]	DA K. A method for stochastic optimization[J]. arXiv:1412.6980, 2014.
[29]	ZHANG J, LI W, OGUNBONA P. Joint geometrical and statistical alignment for visual domain adaptation[C]// Procee-dings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Hawaii, Jul 21-26, 2017. Washing-ton: IEEE Computer Society, 2017: 1859-1867.
[30]	TZENG E, HOFFMAN J, ZHANG N, et al. Deep domain confusion: maximizing for domain invariance[J]. arXiv:1412.3474, 2014.

用于无监督域适应的深度对抗重构分类网络

Deep Adversarial-Reconstruction-Classification Networks for Unsupervised Domain Adaptation

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 9

参考文献 30

相关文章 15

编辑推荐

Metrics

[1]	夏鸿斌, 肖奕飞, 刘渊. 融合自注意力机制的长文本生成对抗网络模型[J]. 计算机科学与探索, 2022, 16(7): 1603-1610.
[2]	申瑞彩, 翟俊海, 侯璎真. 选择性集成学习多判别器生成对抗网络[J]. 计算机科学与探索, 2022, 16(6): 1429-1438.
[3]	姜艺, 胥加洁, 柳絮, 朱俊武. 边缘指导图像修复算法研究[J]. 计算机科学与探索, 2022, 16(3): 669-682.
[4]	孙宇, 魏本征, 刘川, 张魁星, 丛金玉. 融减自动编码器[J]. 计算机科学与探索, 2021, 15(8): 1526-1533.
[5]	王曙燕, 金航, 孙家泽. GAN图像对抗样本生成方法[J]. 计算机科学与探索, 2021, 15(4): 702-711.
[6]	李猛, 李艳玲, 林民. 命名实体识别的迁移学习研究综述[J]. 计算机科学与探索, 2021, 15(2): 206-218.
[7]	费建伟，夏志华，余佩鹏，戴昀书. 人脸合成技术综述[J]. 计算机科学与探索, 2021, 15(11): 2025-2047.
[8]	王富平, 李文楼, 刘颖, 卢津, 公衍超. 结合边缘信息和门卷积的人脸修复算法[J]. 计算机科学与探索, 2021, 15(1): 150-162.
[9]	彭晏飞，高艺，杜婷婷，桑雨，訾玲玲. 生成对抗网络的单图像超分辨率重建方法[J]. 计算机科学与探索, 2020, 14(9): 1612-1620.
[10]	赵鹏飞，李艳玲，林民. 面向迁移学习的意图识别研究进展[J]. 计算机科学与探索, 2020, 14(8): 1261-1274.
[11]	李广丽，滑瑾，袁天，朱涛，邬任重，姬东鸿，张红斌. 基于用户偏好挖掘生成对抗网络的推荐系统[J]. 计算机科学与探索, 2020, 14(5): 803-814.
[12]	武随烁，杨金福，单义，许兵兵. 使用孪生注意力机制的生成对抗网络的研究[J]. 计算机科学与探索, 2020, 14(5): 833-840.
[13]	任豪，刘柏嵩，孙金杨. 面向知识迁移的跨领域推荐算法研究进展[J]. 计算机科学与探索, 2020, 14(11): 1813-1827.
[14]	李俊杰，王茜. 感知相似的图像分类对抗样本生成模型[J]. 计算机科学与探索, 2020, 14(11): 1930-1942.
[15]	梁俊杰，韦舰晶，蒋正锋. 生成对抗网络GAN综述[J]. 计算机科学与探索, 2020, 14(1): 1-17.