基于潜在的低秩约束的不完整模态迁移学习

doi:10.3778/j.issn.1673-9418.2103085

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (12): 2775-2787.DOI: 10.3778/j.issn.1673-9418.2103085

基于潜在的低秩约束的不完整模态迁移学习

徐光生¹^,²^,⁺(), 王士同¹^,²

1.江南大学人工智能与计算机学院，江苏无锡 214122
2.江南大学江苏省媒体设计与软件技术重点实验室，江苏无锡 214122

收稿日期:2021-03-24 修回日期:2021-06-15 出版日期:2022-12-01 发布日期:2021-06-23
通讯作者: +E-mail: 6191610015@stu.jiangnan.edu.cn
作者简介:徐光生（1996—），男，安徽芜湖人，硕士研究生，主要研究方向为人工智能、机器学习。
王士同（1964—），男，江苏扬州人，教授，博士生导师，CCF 会员，主要研究方向为人工智能、模式识别等。
基金资助:
江苏省自然科学基金(BK20191331)

Incomplete Modality Transfer Learning via Latent Low-Rank Constraint

XU Guangsheng¹^,²^,⁺(), WANG Shitong¹^,²

1. School of Artificial Intelligence and Computer Science, Jiangnan University, Wuxi, Jiangsu 214122, China
2. Key Laboratory of Media Design and Software Technology of Jiangsu Province, Jiangnan University, Wuxi, Jiangsu 214122, China

Received:2021-03-24 Revised:2021-06-15 Online:2022-12-01 Published:2021-06-23
About author:XU Guangsheng, born in 1996, M.S. candidate.His research interests include artificial intelligence and machine learning.
WANG Shitong, born in 1964, professor, Ph.D. supervisor, member of CCF. His research interests include artificial intelligence, pattern recognition, etc.
Supported by:
Natural Science Foundation of Jiangsu Province(BK20191331)

摘要/Abstract

摘要：

当数据是多模态时，如果在训练阶段没有足够或完整的目标数据可参与训练，则可能导致训练效果较差甚至失败。为了解决该问题，提出了一个基于潜在的低秩约束的不完整模态迁移学习算法（IMTL）。所提算法通过两方面来解决不完整模态问题：一方面，基于低秩约束子空间框架，引入潜在因素来挖掘目标域中缺失的模态信息，然后借助具有完整模态的辅助数据集，通过跨模态或跨数据集方向的迁移学习来帮助模态或数据集之间的数据对齐；另一方面，利用少量标记目标数据来完成监督信息对齐从而保持目标数据在迁移学习过程中的内在结构。实验结果表明，所提算法较之于传统的迁移学习算法有明显优势；即使对于不完整的目标数据，也可以显著地提高分类性能。

关键词: 迁移学习, 不完整模态, 潜在的低秩约束

Abstract:

When insufficient or incomplete multi-modality data are available in training, the corresponding classi-fication learning may lead to poor training performance or even failure. In order to tackle with this problem, the transfer learning algorithm called IMTL (incomplete modality transfer learning via latent low-rank constraint) is proposed in this paper. The proposed algorithm addresses the incomplete modality problem in two ways. Firstly, latent factors are introduced into a low-rank constrained subspace framework so as to mine missing modality infor-mation on the target domain. With the help of an auxiliary yet complete modality dataset, the proposed cross-modality and cross-dataset transfer learning strategy is used to help align data between modalities or datasets. Sec-ondly, a small amount of labeled target data is used to align the supervision information so as to maintain the internal structure of the target data during the transfer learning. Experimental results show that the proposed algorithm outperforms the previous transfer learning algorithms, and significantly improves the classification performance on the adopted incomplete target datasets.

Key words: transfer learning, incomplete modality, latent low-rank constraint

中图分类号:

TP181

徐光生, 王士同. 基于潜在的低秩约束的不完整模态迁移学习[J]. 计算机科学与探索, 2022, 16(12): 2775-2787.

XU Guangsheng, WANG Shitong. Incomplete Modality Transfer Learning via Latent Low-Rank Constraint[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(12): 2775-2787.

图/表 6

参考文献 37

[1]	PAN S J, YANG Q. A survey on transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(10): 1345-1359. DOI URL
[2]	ZHOU Z H. Learning with unlabeled data and its applic-ation to image retrieval[C]// Proceedings of the 2006 Pacific Rim International Conference on Artificial Intelligence. Berlin, Heidelberg: Springer, 2006: 5-10.
[3]	CHEPLYGINA V, DE BRUIJNE M, PLUIM J P W. Not-so-supervised: a survey of semi-supervised, multi-instance, and transfer learning in medical image analysis[J]. Medical Image Analysis, 2019, 54: 280-296. DOI PMID
[4]	DAI W, XUE G R, QIANG Y, et al. Co-clustering based classification for out-of-domain documents[C]// Proceedings of the 13th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, San Jose, Aug 12-15, 2007. New York: ACM, 2007: 210-219.
[5]	程旸, 蒋亦樟, 钱鹏江, 等. 知识迁移的极大熵聚类算法及其在纹理图像分割中的应用[J]. 智能系统学报, 2017, 12(2): 179-187.
	CHENG Y, JIANG Y Z, QIAN P J, et al. A maximum entropy clustering algorithm based on knowledge transfer and its application to texture image segmentation[J]. CAAI Transactions on Intelligent Systems, 2017, 12(2): 179-187.
[6]	ZHUANG F, LUO P, XIONG H, et al. Exploiting asso-ciations between word clusters and document classes for cross-domain text categorization[J]. Statistical Analysis and Data Mining, 2011, 4(1): 100-114. DOI URL
[7]	BLITZER J, MCDONALD R T, PEREIRA F. Domain adaptation with structural correspondence learning[C]// Proceedings of the 2006 Conference on Empirical Methods in Natural Language Processing, Sydney, Jul 22-23, 2006. Stroudsburg: ACL, 2006: 120-128.
[8]	PAN S J, KWOK J T, YANG Q. Transfer learning via dimensionality reduction[C]// Proceedings of the 23rd AAAI Conference on Artificial Intelligence, Chicago, Jul 13-17, 2008. Menlo Park: AAAI, 2008: 677-682.
[9]	SI S, TAO D, GENG B. Bregman divergence-based regul-arization for transfer subspace learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2009, 22(7): 929-942. DOI URL
[10]	毛发贵, 李碧雯, 沈备军. 基于实例迁移的跨项目软件缺陷预测[J]. 计算机科学与探索, 2016, 10(1): 43-55. DOI URL
	MAO F G, LI B W, SHEN B J. Cross-project software defect prediction based on instance transfer[J]. Journal of Frontiers of Computer Science and Technology, 2016, 10(1): 43-55. DOI URL
[11]	DAI W Y, YANG Q, XUE G R, et al. Boosting for transfer learning[C]// Proceedings of the 24th International Conference on Machine Learning, Corvallis, Jun 20-24, 2007. New York: ACM, 2007: 193-200.
[12]	LIU G, LIN Z, YAN S, et al. Robust recovery of subspace structures by low-rank representation[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2012, 35(1): 171-184. DOI URL
[13]	RAAB C, SCHLEIF F M. Domain adaptation via low-rank basis approximation[J]. arXiv:1907.01343, 2019.
[14]	SHAO M, KIT D, FU Y. Generalized transfer subspace learning through low-rank constraint[J]. International Journal of Computer Vision, 2014, 109(1/2): 74-93. DOI URL
[15]	LIU G C, YAN S C. Latent low-rank representation for subspace segmentation and feature extraction[C]// Procee-dings of the 2011 IEEE International Conference on Comp-uter Vision, Barcelona, Nov 6-13, 2011. Washington: IEEE Computer Society, 2011: 1615-1622.
[16]	DING Z M, SHAO M, FU Y. Latent low-rank transfer subspace learning for missing modality recognition[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec City, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 1192-1198.
[17]	THOPALLI K, ANIRUDH R, THIAGARAJAN J J, et al. Multiple subspace alignment improves domain adaptation[C]// Proceedings of the 2019 IEEE International Conference on Acoustics, Speech and Signal Processing, Brighton, May 12-17, 2019. Piscataway: IEEE, 2019: 3552-3556.
[18]	BELKIN M, NIYOGI P, SINDHWANI V. Manifold regula-rization: a geometric framework for learning from labeled and unlabeled examples[J]. Journal of Machine Learning Research, 2006, 7: 2399-2434.
[19]	LONG M, WANG J, DING G, et al. Adaptation regulariz-ation: a general framework for transfer learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2013, 26(5): 1076-1089. DOI URL
[20]	YU C N J, JOACHIMS T. Learning structural SVMS with latent variables[C]// Proceedings of the 26th Annual Intern-ational Conference on Machine Learning, Montreal, Jun 14-18, 2009. New York: ACM, 2009: 1169-1176.
[21]	LI J, NAJMI A, GRAY R M. Image classification by a two-dimensional hidden Markov model[J]. IEEE Transactions on Signal Processing, 2000, 48(2): 517-533. DOI URL
[22]	DING Z, SHAO M, FU Y. Missing modality transfer learning via latent low-rank constraint[J]. IEEE Transactions on Image Processing, 2015, 24(11): 4322-4334. DOI PMID
[23]	LIU G C, LIN Z C, YU Y. Robust subspace segmentation by low-rank representation[C]// Proceedings of the 27th Int-ernational Conference on Machine Learning, Haifa, Jun 21-24, 2010. Madison: Omnipress, 2010: 663-670.
[24]	RAZZAGHI P, RAZZAGHI P, ABBASI K. Transfer sub-space learning via low-rank and discriminative reconstruction matrix[J]. Knowledge-Based Systems, 2019, 163: 174-185. DOI URL
[25]	DING Z, SHAO M, FU Y. Incomplete multisource transfer learning[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016, 29(2): 310-323. DOI URL
[26]	杨昌健, 邓赵红, 蒋亦樟, 等. 基于迁移学习的癫痫EEG信号自适应识别[J]. 计算机科学与探索, 2014, 8(3): 329-337. DOI URL
	YANG C J, DENG Z H, JIANG Y Z, et al. Adaptive recog-nition of epileptic EEG signals based on transfer learning[J]. Journal of Frontiers of Computer Science and Technology, 2014, 8(3): 329-337.
[27]	PAN S J, TSANG I W, KWOK J T, et al. Domain adaptation via transfer component analysis[J]. IEEE Transa-ctions on Neural Networks, 2010, 22(2): 199-210.
[28]	DING Z, FU Y. Robust multiview data analysis through collective low-rank subspace[J]. IEEE Transactions on Neural Networks and Learning Systems, 2017, 29(5): 1986-1997. DOI URL
[29]	CAI J F, CANDÈS E J, SHEN Z. A singular value thresh-olding algorithm for matrix completion[J]. SIAM Journal on Optimization, 2010, 20(4): 1956-1982. DOI URL
[30]	SHAO M, FU Y. Hierarchical hyperlingual-words for multi-modality face classification[C]// Proceedings of the 10th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, Shanghai, Apr 22-26, 2013. Washington: IEEE Computer Society, 2013: 1-6.
[31]	JHUO I H, LIU D, LEE D T, et al. Robust visual domain adaptation with low-rank reconstruction[C]// Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Jun 16-21, 2012. Washington: IEEE Computer Society, 2012: 2168-2175.
[32]	GONG B Q, SHI Y, SHA F, et al. Geodesic flow kernel for unsupervised domain adaptation[C]// Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition, Providence, Jun 16-21, 2012. Washington: IEEE Computer Society, 2012: 2066-2073.
[33]	FERNANDO B, HABRARD A, SEBBAN M, et al. Unsup-ervised visual domain adaptation using subspace alignment[C]// Proceedings of the 2013 IEEE International Conference on Computer Vision, Sydney, Dec 1-8, 2013. Washington: IEEE Computer Society, 2013: 2960-2967.
[34]	WANG J D, FENG W J, CHEN Y Q, et al. Visual domain adaptation with manifold embedded distribution alignment[C]// Proceedings of the 26th ACM International Conference on Multimedia, Seoul, Oct 22-26, 2018. New York: ACM, 2018: 402-410.
[35]	TAHMORESNEZHAD J, HASHEMI S. Visual domain adaptation via transfer feature learning[J]. Knowledge and Information Systems, 2017, 50(2): 585-605. DOI URL
[36]	BELHUMEUR P N, HESPANHA J P, KRIEGMAN D J. Eigenfaces vs. Fisherfaces: recognition using class specific linear projection[C]// LNCS 1064: Proceedings of the 4th European Conference on Computer Vision, Cambridge, Apr 15-18, 1996. Berlin, Heidelberg: Springer, 1966: 43-58.
[37]	HE X F, NIYOGI P. Locality preserving projections[C]// Advances in Neural Information Processing Systems 16, Vancouver, Dec 8-13, 2003. Cambridge: MIT Press, 2003: 153-160.

Datasets	Subspace methods	DASA	GFK	RDALR	MEDA	MMTL	IMTL
BUAANIR	PCA	76.15±0.55	74.33±0.65	72.86±0.86	46.33±0.33	75.90±0.38	80.72±0.77
	LPP	71.56±0.22	70.86±0.84	68.73±0.84	71.84±0.26	75.72±0.25	78.68±0.02
	LDA	68.64±0.27	67.41±0.89	60.87±0.05	21.61±0.65	74.64±0.37	77.57±0.21
BUAAVIS	PCA	57.73±0.47	57.21±0.88	59.29±0.90	34.41±0.96	56.20±0.28	66.22±0.34
	LPP	54.81±0.93	62.09±0.56	53.76±0.55	48.09±0.05	55.95±0.98	70.25±0.42
	LDA	71.28±0.29	57.18±0.65	45.72±0.54	22.25±0.14	55.33±0.24	68.92±0.03
OuluNIR	PCA	69.12±0.44	72.20±0.23	71.86±0.09	76.43±0.04	72.91±0.27	78.04±0.73
	LPP	72.30±0.92	75.44±0.08	62.70±0.50	78.63±0.05	71.18±0.63	83.04±0.84
	LDA	34.12±0.81	49.63±0.92	43.10±0.60	68.81±0.02	71.89±0.36	79.39±0.32
OuluVIS	PCA	87.33±0.95	86.99±0.11	86.49±0.14	80.78±0.73	87.37±0.39	92.43±0.75
	LPP	86.39±0.87	84.33±0.04	57.26±0.42	86.07±0.02	88.01±0.79	90.47±0.26
	LDA	71.18±0.09	80.71±0.48	49.60±0.19	87.77±0.01	88.14±0.52	92.87±0.46

Datasets	Subspace methods	DASA	GFK	RDALR	MEDA	MMTL	IMTL
BUAANIR	PCA	76.15±0.55	74.33±0.65	72.86±0.86	46.33±0.33	75.90±0.38	80.72±0.77
	LPP	71.56±0.22	70.86±0.84	68.73±0.84	71.84±0.26	75.72±0.25	78.68±0.02
	LDA	68.64±0.27	67.41±0.89	60.87±0.05	21.61±0.65	74.64±0.37	77.57±0.21
BUAAVIS	PCA	57.73±0.47	57.21±0.88	59.29±0.90	34.41±0.96	56.20±0.28	66.22±0.34
	LPP	54.81±0.93	62.09±0.56	53.76±0.55	48.09±0.05	55.95±0.98	70.25±0.42
	LDA	71.28±0.29	57.18±0.65	45.72±0.54	22.25±0.14	55.33±0.24	68.92±0.03
OuluNIR	PCA	69.12±0.44	72.20±0.23	71.86±0.09	76.43±0.04	72.91±0.27	78.04±0.73
	LPP	72.30±0.92	75.44±0.08	62.70±0.50	78.63±0.05	71.18±0.63	83.04±0.84
	LDA	34.12±0.81	49.63±0.92	43.10±0.60	68.81±0.02	71.89±0.36	79.39±0.32
OuluVIS	PCA	87.33±0.95	86.99±0.11	86.49±0.14	80.78±0.73	87.37±0.39	92.43±0.75
	LPP	86.39±0.87	84.33±0.04	57.26±0.42	86.07±0.02	88.01±0.79	90.47±0.26
	LDA	71.18±0.09	80.71±0.48	49.60±0.19	87.77±0.01	88.14±0.52	92.87±0.46

Datasets	Subspace methods	DASA	GFK	RDALR	MEDA	MMTL	IMTL
CMUHR	PCA	18.81±0.55	23.14±0.31	17.01±0.19	48.64±0.31	39.58±0.67	56.51±0.33
	LPP	27.30±0.63	24.60±0.76	18.71±0.06	51.30±0.10	53.89±0.44	60.16±0.47
	LDA	19.01±0.64	21.73±0.39	20.39±0.19	59.40±0.05	49.82±0.41	57.32±0.50
CMULR	PCA	9.69±0.77	11.30±0.73	10.32±0.83	31.36±0.09	17.08±0.18	37.53±0.25
	LPP	11.21±0.50	12.04±0.03	9.39±0.74	29.41±0.41	33.27±0.40	35.31±0.45
	LDA	16.55±0.09	13.81±0.16	16.38±1.02	25.84±0.04	35.40±0.32	38.36±0.80
YaleBHR	PCA	13.77±0.04	12.61±0.55	13.38±0.55	35.63±0.06	27.14±0.42	41.58±0.55
	LPP	17.95±0.15	18.54±0.80	11.65±0.62	35.51±0.19	36.43±0.40	42.73±0.25
	LDA	11.66±0.52	15.03±0.61	15.25±0.24	32.91±0.07	34.26±0.32	39.35±0.73
YaleBLR	PCA	12.52±0.47	10.94±0.18	11.98±0.21	22.92±0.12	14.22±0.22	28.54±0.23
	LPP	11.54±0.80	11.54±0.52	12.94±0.06	28.24±0.58	24.67±0.36	30.69±0.99
	LDA	10.81±0.30	12.15±0.81	17.82±0.97	23.87±0.28	25.88±0.05	29.95±0.41

Datasets	Subspace methods	DASA	GFK	RDALR	MEDA	MMTL	IMTL
CMUHR	PCA	18.81±0.55	23.14±0.31	17.01±0.19	48.64±0.31	39.58±0.67	56.51±0.33
	LPP	27.30±0.63	24.60±0.76	18.71±0.06	51.30±0.10	53.89±0.44	60.16±0.47
	LDA	19.01±0.64	21.73±0.39	20.39±0.19	59.40±0.05	49.82±0.41	57.32±0.50
CMULR	PCA	9.69±0.77	11.30±0.73	10.32±0.83	31.36±0.09	17.08±0.18	37.53±0.25
	LPP	11.21±0.50	12.04±0.03	9.39±0.74	29.41±0.41	33.27±0.40	35.31±0.45
	LDA	16.55±0.09	13.81±0.16	16.38±1.02	25.84±0.04	35.40±0.32	38.36±0.80
YaleBHR	PCA	13.77±0.04	12.61±0.55	13.38±0.55	35.63±0.06	27.14±0.42	41.58±0.55
	LPP	17.95±0.15	18.54±0.80	11.65±0.62	35.51±0.19	36.43±0.40	42.73±0.25
	LDA	11.66±0.52	15.03±0.61	15.25±0.24	32.91±0.07	34.26±0.32	39.35±0.73
YaleBLR	PCA	12.52±0.47	10.94±0.18	11.98±0.21	22.92±0.12	14.22±0.22	28.54±0.23
	LPP	11.54±0.80	11.54±0.52	12.94±0.06	28.24±0.58	24.67±0.36	30.69±0.99
	LDA	10.81±0.30	12.15±0.81	17.82±0.97	23.87±0.28	25.88±0.05	29.95±0.41

基于潜在的低秩约束的不完整模态迁移学习

Incomplete Modality Transfer Learning via Latent Low-Rank Constraint

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 37

相关文章 13

编辑推荐

Metrics

[1]	刘春磊, 陈天恩, 王聪, 姜舒文, 陈栋. 小样本目标检测研究综述[J]. 计算机科学与探索, 2023, 17(1): 53-73.
[2]	林佳伟, 王士同. 用于无监督域适应的深度对抗重构分类网络[J]. 计算机科学与探索, 2022, 16(5): 1107-1116.
[3]	常田, 章宗长, 俞扬. 随机集成策略迁移[J]. 计算机科学与探索, 2022, 16(11): 2531-2536.
[4]	李猛, 李艳玲, 林民. 命名实体识别的迁移学习研究综述[J]. 计算机科学与探索, 2021, 15(2): 206-218.
[5]	赵鹏飞，李艳玲，林民. 面向迁移学习的意图识别研究进展[J]. 计算机科学与探索, 2020, 14(8): 1261-1274.
[6]	任豪，刘柏嵩，孙金杨. 面向知识迁移的跨领域推荐算法研究进展[J]. 计算机科学与探索, 2020, 14(11): 1813-1827.
[7]	么素素，王宝亮，侯永宏. 绝对不平衡样本分类的集成迁移学习算法[J]. 计算机科学与探索, 2018, 12(7): 1145-1153.
[8]	魏彩娜，钱鹏江，奚臣. 域间F-范数正则化迁移谱聚类方法[J]. 计算机科学与探索, 2018, 12(3): 472-483.
[9]	程旸，王士同，杭文龙. 区别性知识利用的迁移分类学习[J]. 计算机科学与探索, 2017, 11(3): 427-437.
[10]	谢丽潇，邓赵红，史荧中，王士同. 面向癫痫EEG自适应识别的迁移径向基神经网络[J]. 计算机科学与探索, 2016, 10(12): 1729-1736.
[11]	毛发贵，李碧雯，沈备军. 基于实例迁移的跨项目软件缺陷预测[J]. 计算机科学与探索, 2016, 10(1): 43-55.
[12]	李翔，朱全银. 联合聚类和评分矩阵共享的协同过滤推荐[J]. 计算机科学与探索, 2014, 8(6): 751-759.
[13]	杨昌健，邓赵红，蒋亦樟，王士同. 基于迁移学习的癫痫EEG信号自适应识别[J]. 计算机科学与探索, 2014, 8(3): 329-337.