Unsupervised Heterogeneous Domain Adaptation with Fuzzy Rule Learning

doi:10.3778/j.issn.1673-9418.2008064

Journal of Frontiers of Computer Science and Technology ›› 2022, Vol. 16 ›› Issue (2): 403-412.DOI: 10.3778/j.issn.1673-9418.2008064

• Artificial Intelligence • Previous Articles Next Articles

Unsupervised Heterogeneous Domain Adaptation with Fuzzy Rule Learning

SUN Wu¹, DENG Zhaohong¹^,²^,³^,⁺(), LOU Qiongdan¹, GU Xin⁴, WANG Shitong¹

1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China
2. Key Laboratory of Computational Neuroscience and Brain-Inspired Intelligence, Ministry of Education, Fudan University, Shanghai 200433, China
3. Zhangjiang Lab, Shanghai 200120, China
4. Jiangsu North Huguang Opto-Electronics Co., Ltd., Wuxi, Jiangsu 214000, China

Received:2020-08-20 Revised:2020-11-03 Online:2022-02-01 Published:2020-11-19
About author:SUN Wu, born in 1995, M.S. candidate. His research interest is interpretability artificial inte-lligence.
DENG Zhaohong, born in 1981, professor, Ph.D. supervisor, distinguished member of CCF. His research interests include interpretability and un-certainty artificial intelligence and its applications.
LOU Qiongdan, born in 1995, Ph.D. candidate. Her research interests include interpretability and uncertainty artificial intelligence and pattern recognition.
GU Xin, born in 1979, Ph.D., senior engineer.His research interests include pattern recognition, artificial intelligence, image processing technology and its application.
WANG Shitong, born in 1964, professor, Ph.D. supervisor. His research interests include artificial intelligence, pattern recognition, etc.
Supported by:
National Natural Science Foundation of China(61772239);Municipal Science and Technology Major Project of “Basic Transformation and Application Research of Brain and Brain-Like Intelligence” in Shanghai(2018SHZDZX01)

基于模糊规则学习的无监督异构领域自适应

孙武¹, 邓赵红¹^,²^,³^,⁺(), 娄琼丹¹, 顾鑫⁴, 王士同¹

1.江南大学人工智能与计算机学院,江苏无锡 214122
2.复旦大学计算神经科学与类脑智能教育部重点实验室,上海 200433
3.张江实验室,上海 200120
4.江苏北方湖光光电有限公司,江苏无锡 214000

通讯作者: + E-mail: dengzhaohong@jiangnan.edu.cn
作者简介:孙武（1995—）,男,江苏兴化人,硕士研究生,主要研究方向为可解释人工智能。
邓赵红（1981—）,男,安徽蒙城人,教授,博士生导师,CCF杰出会员,主要研究方向为可解释和不确定性人工智能及其应用。
娄琼丹（1995—）,女,江苏邳州人,博士研究生,主要研究方向为可解释和不确定性人工智能与模式识别。
顾鑫（1979—）,男,江苏张家港人,博士,高级工程师,主要研究方向为模式识别、人工智能、图像处理技术研究与应用。
王士同（1964—）,男,江苏扬州人,教授,博士生导师,主要研究方向为人工智能、模式识别等。
基金资助:
国家自然科学基金面上项目(61772239);上海市“脑与类脑智能基础转化应用研究”市级重大科技专项(2018SHZDZX01)

Abstract

Abstract:

Heterogeneous domain adaptation is a technique that uses the knowledge of source domain to model the target domain. The source domain and the target domain are semantically related, but their feature spaces are different. Among existing heterogeneous domain adaptive methods, most of them belong to semi-supervised methods, which require some labeled samples in the target domain. However, this kind of dataset is rare in many heter-ogeneous adaptive tasks. In order to solve the above problem, this paper proposes a new unsupervised heter-ogeneous domain adaptive algorithm based on fuzzy rule learning. On the one hand, by introducing the TSK fuzzy system, the proposed method learns two feature transformation matrices corresponding to the source domain and the target domain respectively. By learning two feature transformation matrices, the source domain and the target domain are projected into a common feature subspace. On the other, in order to reduce the information loss caused by feature transformation, the proposed algorithm adopts a variety of information preservation strategies and maximizes the correlation between the transformed source domain data and target domain data. Through experi-ments on domain adaptive datasets, the results show that the proposed algorithm has certain advantages over the existing heterogeneous domain adaptive methods.

Key words: fuzzy rule learning, TSK fuzzy system, information preserving, heterogeneous domain adaptation

摘要：

异构领域自适应是一种借助源域知识为语义相关但特征空间不同的目标域建模的技术。现有的异构领域自适应方法大多属于半监督方法,这些方法要求目标域中存在一部分已标记样本,然而这种数据集在很多异构领域自适应任务中是稀缺的。为了解决上述问题,提出了一种新的基于模糊规则学习的无监督异构领域自适应算法。一方面,该方法基于TSK模糊系统的规则学习分别对源域和目标域进行特征学习,通过学习两个特征变换矩阵将源域和目标域投影到一个公共特征子空间;另一方面,为了减少因特征变换所造成的信息损失,该算法采取了多种信息保持策略,并且最大化公共特征子空间中源域数据和目标域数据之间的相关性。通过在几个真实领域自适应数据集上进行实验,验证了所提算法相对于现有的异构领域自适应方法具有一定的优越性。

关键词: 模糊规则学习, TSK模糊系统, 信息保持, 异构领域自适应

CLC Number:

TP181

SUN Wu, DENG Zhaohong, LOU Qiongdan, GU Xin, WANG Shitong. Unsupervised Heterogeneous Domain Adaptation with Fuzzy Rule Learning[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 403-412.

孙武, 邓赵红, 娄琼丹, 顾鑫, 王士同. 基于模糊规则学习的无监督异构领域自适应[J]. 计算机科学与探索, 2022, 16(2): 403-412.

Figures/Tables 9

Fig.1 Feature transformation based on TSK-FS

Table 1 Statistical information of datasets

数据集		样本数	维度(SURF/DeCAF)	类别数
Office-Caltech	A(Amazon)	958	800/4 096	10
	D(DSLR)	157	800/4 096
	W(Webcam)	295	800/4 096
	C(Caltech)	1 123	800/4 096
Wiki	Img	500	128	5
Wiki	Txt	500	10	5
Reuters	English	18 758	227	6
	French	18 758	246
	German	18 758	284
	Italian	18 758	209
	Spanish	18 758	162

Table 2 Parameter settings of algorithms

算法	参数设置
LinearCCA	$C = {2 - 15, 2 - 14, ⋯, 29}$
CTSVM	$C = {2 - 15, 2 - 14, ⋯, 29}$
CDLS	$T = 5$ , $δ = 0.5$
FUHDA-noSP	$K = 3$ , $T = 15$ , $α = {2 - 9, 2 - 8, ⋯, 29}$ , $μ = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$
FUHDA-noTP	$K = 3$ , $T = 15$ , $α = {2 - 9, 2 - 8, ⋯, 29}$ , $β = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$
FUHDA-noCCA	$K = 3$ , $T = 15$ , $β = {2 - 9, 2 - 8, ⋯, 29}$ , $μ = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$
FUHDA	$K = 3$ , $T = 15$ , $α = {2 - 9, 2 - 8, ⋯, 29}$ , $β = {2 - 9, 2 - 8, ⋯, 29}$ , $μ = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$

Table 2 Parameter settings of algorithms

算法	参数设置
LinearCCA	$C = {2 - 15, 2 - 14, ⋯, 29}$
CTSVM	$C = {2 - 15, 2 - 14, ⋯, 29}$
CDLS	$T = 5$ , $δ = 0.5$
FUHDA-noSP	$K = 3$ , $T = 15$ , $α = {2 - 9, 2 - 8, ⋯, 29}$ , $μ = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$
FUHDA-noTP	$K = 3$ , $T = 15$ , $α = {2 - 9, 2 - 8, ⋯, 29}$ , $β = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$
FUHDA-noCCA	$K = 3$ , $T = 15$ , $β = {2 - 9, 2 - 8, ⋯, 29}$ , $μ = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$
FUHDA	$K = 3$ , $T = 15$ , $α = {2 - 9, 2 - 8, ⋯, 29}$ , $β = {2 - 9, 2 - 8, ⋯, 29}$ , $μ = {2 - 9, 2 - 8, ⋯, 29}$ , $λ = 1$

Table 3 Accuracy of algorithms on Office-Caltech dataset %

Transfer tasks	Linear CCA	CTSVM	CDLS	FUHDA-noCCA	FUHDA-noSP	FUHDA-noTP	FUHDA
A_S2D	75.55	71.47	93.68	28.18	88.83	87.37	93.84
A_D2S	66.14	63.32	65.21	18.48	48.33	59.29	68.27
C_S2D	47.86	45.45	88.79	37.40	83.97	85.40	88.87
C_D2S	39.57	42.78	53.96	19.32	31.61	45.86	52.00
D_S2D	17.31	57.69	77.17	27.39	99.36	22.93	99.36
D_D2S	26.92	57.69	60.63	25.48	45.22	39.49	85.99
W_S2D	53.06	72.45	97.07	32.88	99.66	73.56	99.66
W_D2S	41.84	56.12	74.48	32.88	54.24	64.41	85.76
Average	46.03	58.37	76.37	27.75	68.90	59.79	84.22

Table 4 Accuracy of algorithms on Wiki dataset %

Transfer tasks	Linear CCA	CTSVM	CDLS	FUHDA-noCCA	FUHDA-noSP	FUHDA-noTP	FUHDA
Txt2Img	39.80	43.40	44.20	51.40	55.20	55.00	60.40
Img2Txt	80.00	78.40	92.83	30.00	91.40	91.60	95.20
Average	59.90	60.90	68.51	40.70	73.30	73.30	77.80

Table 5 Accuracy of algorithms on Reuters dataset %

Transfer tasks	Linear CCA	CTSVM	CDLS	FUHDA-noCCA	FUHDA-noSP	FUHDA-noTP	FUHDA
EN2SP	33.00	49.66	49.86	23.80	34.96	42.87	54.82
FR2SP	33.57	33.05	50.67	27.20	46.16	46.21	53.60
GR2SP	65.08	59.23	50.36	27.20	30.28	47.43	57.24
IT2SP	19.30	21.37	50.83	23.09	32.16	36.35	62.29
Average	37.74	40.83	50.43	25.32	35.89	43.22	56.99

Fig.2 Rule number analysis

Fig.3 Convergence analysis

Fig.4 Parameter sensitivity analysis

References 36

[1]	PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(10):1345-1359. DOI URL
[2]	LU J, BEHBOOD V, HAO P, et al. Transfer learning using computational intelligence: a survey[J]. Knowledge-Based Systems, 2015, 80:14-23. DOI URL
[3]	DAY O, KHOSHGOFTAAR T M. A survey on heterogen-eous transfer learning[J]. Journal of Big Data, 2017, 4(1):29. DOI URL
[4]	朱应钊. 异构迁移学习研究综述[J]. 电信科学, 2020, 36(3):100-110.
	ZHU Y Z. Review on heterogeneous transfer learning[J]. Telecommunication Science, 2020, 36(3):100-110.
[5]	AYTAR Y, ZISSERMAN A. Tabula rasa: model transfer for object category detection[C]//Proceedings of the 2011 IEEE International Conference on Computer Vision, Barcelona, Nov 6-13, 2011. Washington: IEEE Computer Society, 2011: 2252-2259.
[6]	YANG J, YAN R, HAUPTMANN A G. Cross-domain video concept detection using adaptive SVMs[C]//Proceedings of the 15th International Conference on Multimedia 2007, Aug-sburg, Sep 24-29, 2007. New York: ACM, 2007: 188-197.
[7]	BERGAMO A, TORRESANI L. Exploiting weakly-labeled Web images to improve object classification: a domain adaptation approach[C]//Proceedings of the 24th Annual Conference on Neural Information Processing Systems, Van-couver, Dec 6-9, 2010. Red Hook: Curran Associates, 2010: 181-189.
[8]	PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE Transactions on Neural Networks, 2010, 22(2):199-210. DOI URL
[9]	LONG M S, WANG J M, DING G G, et al. Transfer feature learning with joint distribution adaptation[C]//Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 2200-2207.
[10]	GONG B Q, SHI Y, SHA F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]//Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Jun 16-21, 2012. Washington: IEEE Computer Society, 2012: 2066-2073.
[11]	LI J J, LU K, HUANG Z, et al. Transfer independently together: a generalized framework for domain adaptation[J]. IEEE Transactions on Cybernetics, 2018, 49(6):2144-2155. DOI URL
[12]	SI S, TAO D C, GENG B. Bregman divergence-based regu-larization for transfer subspace learning[J]. IEEE Transac-tions on Knowledge and Data Engineering, 2009, 22(7):929-942.
[13]	ZHUANG F Z, CHENG X H, LUO P, et al. Supervised representation learning: transfer learning with deep auto-encoders[C]//Proceedings of the 24th International Joint Conference on Artificial Intelligence, Buenos Aires, Jul 25-31, 2015. Menlo Park: AAAI, 2015: 4119-4125.
[14]	SHEN J, QU Y R, ZHANG W N, et al. Wasserstein distance guided representation learning for domain adaptation[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 4058-4065.
[15]	SHI X X, LIU Q, FAN W, et al. Transfer learning on heterogenous feature spaces via spectral transformation[C]//Proceedings of the 10th IEEE International Conference on Data Mining, Sydney, Dec 14-17, 2010. Washington: IEEE Computer Society, 2010: 1049-1054.
[16]	WANG C, MAHADEVAN S. Heterogeneous domain adapta-tion using manifold alignment[C]//Proceedings of the 22nd International Joint Conference on Artificial Intelligence, Bar-celona, Jul 16-22, 2011. Menlo Park: AAAI, 2011: 1541-1546.
[17]	KULIS B, SAENKO K, DARRELL T. What you saw is not what you get: domain adaptation using asymmetric kernel transforms[C]//Proceedings of the 24th IEEE Conference on Computer Vision and Pattern Recognition, Colorado Springs, Jun 20-25, 2011. Washington: IEEE Computer So-ciety, 2011: 1785-1792.
[18]	LI W, DUAN L X, XU D, et al. Learning with augmented features for supervised and semi-supervised heterogeneous domain adaptation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2013, 36(6):1134-1148. DOI URL
[19]	HOFFMAN J. Efficient learning of domain-invariant image representations[J]. arXiv:1301.3224, 2013.
[20]	YEH Y R, HUANG C H, WANG Y C F. Heterogeneous domain adaptation and classification by exploiting the corr-elation subspace[J]. IEEE Transactions on Image Processing, 2014, 23(5):2009-2018. DOI URL
[21]	SHEN C, GUO Y H. Unsupervised heterogeneous domain adaptation with sparse feature transformation[C]// Proceedings of the 10th Asian Conference on Machine Learning, Beijing, Nov 14-16, 2018: 375-390.
[22]	LIU F, LU J, ZHANG G. Unsupervised heterogeneous domain adaptation via shared fuzzy equivalence relations[J]. IEEE Transactions on Fuzzy Systems, 2018, 26(6):3555-3568. DOI URL
[23]	BALAKRISHNAMA S, GANAPATHIRAJU A. Linear discriminant analysis—a brief tutorial[J]. Institute for Signal and Information Processing, 1998, 18:1-8.
[24]	WOLD S, ESBENSEN K, GELADI P. Principal component analysis[J]. Chemometrics and Intelligent Laboratory Systems, 1987, 2:37-52. DOI URL
[25]	THOMPSON B. Canonical correlation analysis[M]//EVERITT B, HOWELL D. Encyclopedia of Statistics in Behavioral Science. New York: John Wiley & Sons, Inc., 2005.
[26]	TAKAGI T, SUGENO M. Fuzzy identification of systems and its applications to modeling and control[J]. IEEE Transac-tions on Systems, Man, and Cybernetics, 1985(1):116-132.
[27]	DENG Z H, JIANG Y Z, CHOI K S, et al. Knowledge-leverage-based TSK fuzzy system modeling[J]. IEEE Tran-sactions on Neural Networks and Learning Systems, 2013, 24(8):1200-1212.
[28]	DENG Z H, JIANG Y Z, CAO L, et al. Knowledge-leverage based TSK fuzzy system with improved knowledge transfer[C]//Proceedings of the 2014 IEEE International Conference on Fuzzy Systems. Piscataway: IEEE, 2014: 178-185.
[29]	ZUO H, ZHANG G Q, PEDRYCZ W, et al. Fuzzy regression transfer learning in Takagi-Sugeno fuzzy models[J]. IEEE Transactions on Fuzzy Systems, 2016, 25(6):1795-1807. DOI URL
[30]	XU P, DENG Z H, WANG J, et al. Transfer representation learning with TSK fuzzy system[J]. IEEE Transactions on Fuzzy Systems, 2021, 29(3):649-663. DOI URL
[31]	蒋亦樟, 邓赵红, 王士同. ML型迁移学习模糊系统[J]. 自动化学报, 2012, 38(9):1393-1409.
	JIANG Y Z, DENG Z H, WANG S T. Mamdani-Larsen type transfer learning fuzzy system[J]. Acta Automatica Sinica, 2012, 38(9):1393-1409.
[32]	MENDEL J M. Fuzzy logic systems for engineering: a tutorial[J]. Proceedings of the IEEE, 1995, 83(3):345-377. DOI URL
[33]	关庆, 邓赵红, 王士同. 改进的模糊C-均值聚类算法[J]. 计算机工程与应用, 2011, 47(10):27-29.
	GUAN Q, DENG Z H, WANG S T. Improved fuzzy C-means clustering algorithm[J]. Computer Engineering and Applications, 2011, 47(10):27-29.
[34]	SU T, DY J G. In search of deterministic methods for initializing K-means and Gaussian mixture clustering[J]. Inte-lligent Data Analysis, 2007, 11(4):319-338.
[35]	HARDOON D R, SZEDMAK S, SHAWE-TAYLOR J. Can-onical correlation analysis: an overview with application to learning methods[J]. Neural Computation, 2004, 16(12):2639-2664. DOI URL
[36]	TSAI Y H, YEH Y R, WANG Y C. Learning cross-domain landmarks for heterogeneous domain adaptation[C]//Pro-ceedings of the 2016 Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 5081-5090.

Unsupervised Heterogeneous Domain Adaptation with Fuzzy Rule Learning

基于模糊规则学习的无监督异构领域自适应

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 36

Related Articles 3

Recommended Articles

Metrics

[1]	XU Peng, DENG Zhaohong, WANG Jun, WANG Shitong. Joint Information Preservation for Heterogeneous Domain Adaptation [J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(7): 1183-1193.
[2]	ZHANG Chunxiang, WANG Jun, ZHANG Jiaxu, DENG Zhaohong, PAN Xiang, WANG Shitong. Novel TSK Modeling Method with Joint Group Sparse Learning for Autism Aided Diagnosis [J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(12): 2083-2093.
[3]	CAO Ya, DENG Zhaohong, WANG Shitong. TSK Fuzzy System Model with Monotonic Constraints [J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(9): 1487-1495.