具有类间差异约束的多对抗深度域适应模型

doi:10.3778/j.issn.1673-9418.2108111

摘要/Abstract

摘要： 为实现目标域样本能够与源域中同类样本准确对齐，并在保证样本准确识别率的条件下进一步提高不同类别样本特征间的可区分性，提出了一种带有类间差异约束的域适应模型。首先，该模型采用深度卷积神经网络对源域样本进行了有监督学习，并在训练过程中基于提出的类间差异测量函数对源域样本特征加以类间差异性约束；其次，该模型采用了多对抗域鉴别网络结构，其中提出了一种目标域样本伪标签计算方法，从而将无标签的样本指定到合理的域鉴别网络进行训练；最后，通过最小化分类损失与最大化域鉴别损失，获得最优特征提取器与特征分类器。实验结果表明，对于4种数据集，提出的模型在目标域上平均识别准确率可以达到0.860，同类间的平均距离、不同类间的平均距离、目标域中样本错误识别率相对于改进前分别降低0.003，提升0.065，降低0.025，从而验证了提出模型的性能得到了明显提升。

关键词: 迁移学习, 深度域适应, 类间差异, 多对抗网络

Abstract: To align the samples in the target domain to the samples of the same class in the source domain accurately, and improve the feature distinguishability between different classes further while ensuring recognition accuracy, a domain adaptation model with inter-class difference constraint is proposed. Firstly, a deep neural network is used to perform supervised learning on source domain samples, and the inter-class difference constraint is imposed on the sample features in the source domain based on the proposed inter-class difference measurement function. Then, multi-adversarial domain discrimination network is adopted, in which a pseudo label solving method is proposed to assign the unlabeled samples to the reasonable domain discriminators for training. Finally, the best feature extractor and feature classifier can be obtained through minimizing the classification loss and maximizing the domain discrimination loss. Experimental results show that for the four datasets, the average recognition accuracy of the proposed model on the target domain can reach 0.860, and the average distance between the same classes, the average distance between different classes, the error recognition rate in the target domain can be reduced by 0.003, improved by 0.065, and reduced by 0.025, respectively, which is verified that the performance of proposed model is improved significantly.

Key words: transfer learning, deep domain adaptation, inter-class difference, multiple adversarial networks

马娜, 温廷新, 贾旭. 具有类间差异约束的多对抗深度域适应模型[J]. 计算机科学与探索, 2023, 17(5): 1168-1179.

MA Na, WEN Tingxin, JIA Xu. Multiple Adversarial Deep Domain Adaptation Model with Inter-class Difference Constraint[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(5): 1168-1179.

参考文献

[1] 范苍宁, 刘鹏, 肖婷, 等. 深度域适应综述：一般情况与复杂情况[J]. 自动化学报, 2021, 47(3): 515-548.
FAN C N, LIU P, XIAO T, et al. A review of deep domain adaptation: general situation and complex situation[J]. Acta Automatica Sinica, 2021, 47(3): 515-548.
[2] LONG M S, ZHU H, WANG J M, et al. Unsupervised domain adaptation with residual transfer networks[C]//Proceedings of the 30th International Conference on Neural Information Processing Systems, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates, 2016: 136-144.
[3] TZENG E, HOFFMAN J, ZHANG N, et al. Deep domain confusion: maximizing for domain invariance[J]. arXiv:1412.3474, 2014.
[4] LONG M S, CAO Y, WANG J M, et al. Learning transferable features with deep adaptation networks[C]//Proceedings of the 32nd International Conference on Machine Learning, Lile, Jul 6-11, 2015: 97-105.
[5] LONG M S, ZHU H, WANG J M, et al. Deep transfer learning with joint adaptation networks[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, Aug 6-11, 2017: 2208-2217.
[6] YAN H L, DING Y K, LI P H, et al. Mind the class weight bias: weighted maximum mean discrepancy for unsupervised domain adaptation[C]//Proceedings of the 2017 IEEE Con-ference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 945-954.
[7] SHEN J, QU Y R, ZHANG W N, et al. Wasserstein distance guided representation learning for domain adaptation[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 4058-4065.
[8] ZHOU G Y, HUANG J X. Modeling and mining domain shared knowledge for sentiment analysis[J]. ACM Transactions on Information Systems, 2017, 36(2): 1-36.
[9] PENG X C, SAENKO K. Synthetic to real adaptation with generative correlation alignment networks[C]//Proceedings of the 2018 IEEE Winter Conference on Applications of Computer Vision, Lake Tahoe, Mar 12-15, 2018. Washington: IEEE Computer Society, 2018: 1982-1991.
[10] SUN B C, SAENKO K. Deep CORAL: correlation alignment for deep domain adaptation[C]//LNCS 9915: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 8-16, 2016. Cham: Springer, 2016: 443-450.
[11] CHEN C, CHEN Z H, JIANG B Y, et al. Joint domain alignment and discriminative feature learning for unsupervised deep domain adaptation[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 3296-3303.
[12] LI Y J, SWERSKY K, ZEMEL R S. Generative moment matching networks[C]//Proceedings of the 32nd International Conference on Machine Learning, Lille, Jul 6-11, 2015: 1718-1727.
[13] ZELLINGER W, GRUBINGER T, LUGHOFER E, et al. Central moment discrepancy (CMD) for domain-invariant representation learning[J]. arXiv:1702.08811, 2017.
[14] ROZANTSEV A, SALZMANN M, FUA P. Beyond sharing weights for deep domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41(4): 801-814.
[15] 许夙晖, 慕晓冬, 柴栋, 等. 基于极限学习机参数迁移的域适应算法[J]. 自动化学报, 2018, 44(2): 311-317.
XU S H, MU X D, CHAI D, et al. Domain adaption algorithm with ELM parameter transfer[J]. Acta Automatica Sinica, 2018, 44(2): 311-317.
[16] WU S, ZHONG J, CAO W M, et al. Improving domain-specific classification by collaborative learning with adaptation networks[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 5450-5457.
[17] GONG R, LI W, CHEN Y H, et al. DLOW: domain flow for adaptation and generalization[C]//Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway:IEEE, 2019: 2477-2486.
[18] YANG Z L, ZHAO J J, DHINGRA B, et al. GLoMo: unsupervised learning of transferable relational graphs[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, Dec 3-8, 2018. Cambridge: MIT Press, 2018: 8964-8975.
[19] XU X, ZHOU X, VENKATESAN R, et al. d-SNE: domain adaptation using stochastic neighborhood embedding[C]// Proceedings of the 2019 IEEE Conference on Computer Vision and Pattern Recognition, Long Beach, Jun 16-20, 2019. Piscataway: IEEE, 2019: 2497-2506.
[20] DENG Z J, LUO Y C, ZHU J. Cluster alignment with a teacher for unsupervised domain adaptation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 9943-9952.
[21] ZHANG Y H, ZHANG Y, WEI Y, et al. Fisher deep domain adaptation[C]//Proceedings of the 2020 SIAM International Conference on Data Mining, Cincinnati, May 7-9, 2020. Philadelphia: SIAM, 2020: 469-477.
[22] PENG X C, BAI Q X, XIA X D, et al. Moment matching for multi-source domain adaptation[C]//Proceedings of the 2019 IEEE/CVF International Conference on Computer Vision, Seoul, Oct 27-Nov 2, 2019. Piscataway: IEEE, 2019: 1406-1415.
[23] WANG M, DENG W H. Deep visual domain adaptation: a survey[J]. Neurocomputing, 2018, 312: 135-153.
[24] GANIN Y, LEMPITSKY V S. Unsupervised domain adaptation by backpropagation[C]//Proceedings of the 32nd International Conference on Machine Learning, Lille, Jul 6-11, 2015: 1180-1189.
[25] PEI Z Y, CAO Z J, LONG M S, et al. Multi-adversarial domain adaptation[C]//Proceedings of the 32nd AAAI Con-ference on Artificial Intelligence, the 30th Innovative App-lications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 3934-3941.
[26] TZENG E, HOFFMAN J, SAENKO K, et al. Adversarial discriminative domain adaptation[C]//Proceedings of the 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 2962-2971.
[27] SANKARANARAYANA S, BALAJI Y, CASTILLO C D, et al. Generate to adapt: aligning domains using generative adversarial networks[C]//Proceedings of the 2018 IEEE Conference on Computer Vision and Pattern Recognition, Salt Lake City, Jun 18-23, 2018. Washington: IEEE Computer Society, 2018: 8503-8512.
[28] LONG M S, CAO Z J, WANG J M, et al. Conditional adversarial domain adaptation[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montréal, Dec 2-8, 2018. Cambridge: MIT Press, 2018: 1647-1657.
[29] WANG X M, LI L, YE W R, et al. Transferable attention for domain adaptation[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park:AAAI, 2019: 5345-5352.
[30] SU J C, TSAI Y H, SOHN K, et al. Active adversarial domain adaptation[C]//Proceedings of the 2020 IEEE Winter Conference on Applications of Computer Vision, Snowmass Village, Mar 1-5, 2020. Piscataway: IEEE, 2020: 728-737.
[31] GLOROT X, BORDES A, BENGIO Y. Domain adaptation for largescale sentiment classification: a deep learning approach[C]//Proceedings of the 28th International Conference on Machine Learning, Bellevue, Jun 28-Jul 2, 2011. Madison: Omnipress, 2011: 513-520.
[32] GHIFARY M, KLEIJN W B, ZHANG M J, et al. Deep reconstruction-classification networks for unsupervised domain adaptation[C]//LNCS 9908: Proceedings of the 14th European Conference on Computer Vision, Amsterdam, Oct 11-14, 2016. Cham: Springer, 2016: 597-613.
[33] ZHUANG F Z, CHENG X H, LUO P, et al. Supervised representation learning: transfer learning with deep autoen-coders[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Jul 25-31, 2015. Menlo Park: AAAI, 2015: 4119-4125.
[34] TSAI J C, CHIEN J T. Adversarial domain separation and adaptation[C]//Proceedings of the 27th IEEE International Workshop on Machine Learning for Signal Processing, Tokyo, Sep 25-28, 2017. Piscataway: IEEE, 2017: 1-6.
[35] ZHU J Y, PARK T, ISOLA P, et al. Unpaired image-to-image translation using cycle-consistent adversarial networks[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2242-2251.
[36] YI Z L, ZHANG H, TAN P, et al. DualGAN: unsupervised dual learning for image-to-image translation[C]//Proceedings of the 2017 IEEE International Conference on Computer Vision, Venice, Oct 22-29, 2017. Washington: IEEE Computer Society, 2017: 2868-2876.
[37] SAENKO K, KULIS B, FRITZ M, et al. Adapting visual category models to new domains[C]//LNCS 6314: Proceedings of the 11th European Conference on Computer Vision, Crete, Sep 5-11, 2010. Berlin, Heidelberg: Springer, 2010: 213-226.
[38] GONG B Q, SHI Y, SHA F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Jun 16-21, 2012. Washington: IEEE Computer Society, 2012: 2066-2073.
[39] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[40] LECUN Y, BOTTOU L. Gradient-based learning applied to document recognition[J]. Proceedings of the IEEE, 1998, 86(11): 2278-2324.
[41] LECUN Y, MATAN O, BOSER B E, et al. Handwritten zip code recognition with multilayer networks[C]//Proceedings of the 10th IAPR International Conference on Pattern Recognition, Atlantic City, Jun 16-21, 1990. Piscataway:IEEE, 1990: 35-40.
[42] NETZER Y, WANG T, COATES A, et al. Reading digits in natural images with unsupervised feature learning[C]//Proceedings of the 2011 NIPS Workshop on Deep Learning and Unsupervised Feature Learning, Granada, Dec 12-17 , 2011. Red Hook: Curran Associates, 2011: 1-9.
[43] XIE S A, ZHENG Z B, CHEN L, et al. Learning semantic representations for unsupervised domain adaptation[C]//Proceedings of the 35th International Conference on Machine Learning, Stockholmsm?ssan, Jul 10-15, 2018: 5419-5428.