Cross-Domain Recommendation Algorithm Combining Multi-personalized Bridges and Self-supervised Learning

doi:10.3778/j.issn.1673-9418.2305022

Abstract

Abstract: A cross-domain recommendation algorithm combining multi-personalized bridges and self-supervised learning (MS-PTUPCDR) is proposed for users with less project interaction in the target domain in the cross-domain recommendation system. Firstly, a variational bipartite graph encoder is added to the target domain, and a variational inference framework is used to generate potential variables. The target domain user representation aggregates their isomor-phic neighbor information. Secondly, the user??s single preference bridge is extended to the user??s multi-personalized preference bridge, the user's transferable user factors in the multi-source domain are transferred to the target domain, and the multi-head attention mechanism is added to the target domain to fuse the user's potential factors from different source domains as the auxiliary task of self-supervised learning. Finally, this paper aggregates user neighbor factors and the fused user multi-source domain transfer user factors for self-supervised learning. In the target domain, the project score of the target domain is predicted by the dot product of the user factor and the project factor of the target domain after the user's supervised learning. The algorithm is tested on two datasets, Amazon and MovieLens, and the results show that the algorithm outperforms the comparative baseline algorithm in terms of MAE and RMSE evaluation metrics. Compared with the optimal comparative baseline algorithm on both datasets, the MAE is improved by 1.96% on average, and the RMSE is improved by 1.92% on average, which verifies the effectiveness of the algorithm.

Key words: cross-domain recommendation, multi-personalized preference bridges for users, multi-head attention mechanism, self-supervised learning, variational bipartite graph encoder

摘要： 针对跨域推荐系统中目标域中项目交互较少的用户，提出一种融合多个性化桥和自监督学习的跨域推荐算法（MS-PTUPCDR）。首先，在目标域加入变分二部图编码器，使用变分推理框架生成潜在变量，目标域用户表示聚合其同构邻居信息。其次，将用户单一偏好桥扩展为用户多个性化偏好桥，将用户在多源域可转移的用户因子转移到目标域，在目标域加入多头注意力机制融合分别来自不同源域转换的用户潜在因子作为自监督学习的辅助任务。最后，在目标域中将聚合用户邻居因子和融合后的用户多源域转移用户因子进行自监督学习。在目标域通过用户自监督学习后的用户因子和目标域项目因子点积进行目标域项目评分预测。算法在Amazon和MovieLens两个数据集上进行实验，结果表明算法在MAE和RMSE两个评价指标上优于对比基线算法，在两个数据集上与最优对比基线算法相比，MAE平均提升1.96%，RMSE平均提升1.92%，验证了算法的有效性。

WANG Yonggui, LIU Danni. Cross-Domain Recommendation Algorithm Combining Multi-personalized Bridges and Self-supervised Learning[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(7): 1792-1805.

王永贵, 刘丹妮. 融合多个性化桥和自监督学习的跨域推荐算法[J]. 计算机科学与探索, 2024, 18(7): 1792-1805.

References

[1] 赵梦媛, 黄晓雯, 桑基韬, 等. 对话推荐算法研究综述[J]. 软件学报, 2022,?33(12):?4616-4643.
ZHAO M Y, HUANG X W, SANG J T, et al. Survey on conversational recommendation algorithms[J]. Journal of Software, 2022, 33(12): 4616-4643.
[2] ZANG T Z, ZHU Y M, LIU H B, et al. A survey on cross-domain recommendation: taxonomies, methods, and future directions[J]. ACM Transactions on Information Systems,2023, 41(2): 1-39.
[3] WANG D, LIANG Y, XUD, et al. A content-based recommender system for computer science publication[J]. Knowledge-Based Systems, 2018, 157: 1-9.
[4] REDDY S R S, NALLURI S, KUNISETI S, et al. Content-based movie recommendation system using genre correlation[M]//Smart Intelligent Computing and Applications. Singapore: Springer, 2019: 391-397.
[5] FU M, QU H, YI Z, et al. A novel deep learning-based collaborative filtering model for recommendation system[J].IEEE Transactions on Cybernetics, 2018, 49(3): 1084-1096.
[6] XUE F, HE X, WANG X, et al. Deep item-based collaborative filtering for top-n recommendation[J]. ACM Transactions on Information Systems, 2019, 37(3): 1-25.
[7] WU D. Music personalized recommendation system based on hybrid filtration[C]//Proceedings of the 2019 International Conference on Intelligent Transportation Big Data and Smart City. Piscataway: IEEE, 2019: 430-433.
[8] DHRUV A, KAMATH A, POWAR A, et al. Artist recommendation system using hybrid method:a novel approach[M]//Emerging Research in Computing, Information, Communication and Applications. Singapore: Springer, 2019: 527-542.
[9] BANDYOPADHYAY S, THAKUR S S. Product prediction and recommendation in e-commerce using collaborative filtering and artificial neural networks: a hybrid approach[M]//Intelligent Computing Paradigm: Recent Trends. Singapore: Springer, 2020: 59-67.
[10] 任豪, 刘柏嵩, 孙金杨. 面向知识迁移的跨领域推荐算法研究进展[J]. 计算机科学与探索, 2020, 14(11): 1813-1827.
REN H, LIU B S, SUN J Y. Advances and perspectives on knowledge transfer based cross-domain recommendation[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(11): 1813-1827.
[11] 陶鸿, 吴国栋, 孙成, 等. 跨领域推荐研究进展[J]. 长春师范大学学报, 2019, 38(12): 44-54.
TAO H, WU G D, SUN C, et al. Research progress in cross-domain recommendation[J]. Journal of Changchun Normal University, 2019, 38(12): 44-54.
[12] ZHU F, WANG Y, CHEN C C, et al. A deep framework for cross-domain and cross-system recommendations[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018: 3711-3717.
[13] KANG S K, HWANG J Y, LEE D H, et al. Semi-supervised learning for cross-domain recommendation to cold-start users[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management. New York: ACM, 2019: 1563-1572.
[14] ZHU Y C, GE K K, ZHUANG F Z, et al. Transfer-meta framework for cross-domain recommendation to cold-start users[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2021: 1813-1817.
[15] ZHU Y C, TANG Z W, LIU Y D, et al. Personalized transfer of user preferences for cross-domain recommendation[C]//Proceedings of the 15th ACM International Conference on Web Search and Data Mining. New York: ACM, 2022: 1507-1515.
[16] WANG T X, ZHUANG F Z, ZHANG Z Q, et al. Low-dimensional alignment for cross-domain recommendation[C]//Proceedings of the 30th ACM International Conference on Information and Knowledge Management. New York: ACM, 2021: 3508-3512.
[17] 孙爱晶, 王国庆. 邻居关系感知的图卷积网络推荐模型[J]. 计算机工程与应用, 2023, 59(9): 112-122.
SUN A J, WANG G Q. Neighbor relation-aware graph convolutional network for recommendation[J]. Computer Engineering and Applications, 2023, 59(9): 112-122.
[18] KIPF T N, WELLING M. Semi-supervised classifification with graph convolutional networks[EB/OL]. [2023-03-21].https://arxiv.org/abs/1609.02907v4.
[19] WANG X, HE X, WANG M, et al. Neural graph collaborative filtering[C]//Proceedings of the 2019 ACM International Conference on Research on Development in Information Retrieval. New York: ACM, 2019: 165-174.
[20] YING R, HE R, CHEN K, et al. Graph convolutional neural networks for web-scale recommender systems[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York:ACM, 2018: 974-983.
[21] CAO J X, SHENG J W, CONG X, et al. Cross-domain recommendation to cold-start users via variational information bottleneck[EB/OL]. [2023-03-12]. https://arxiv.org/abs/2203.16863v1.
[22] KINGMA D P, WELLING M. Auto-encoding variational Bayes[C]//Proceedings of the 2014 International Conference on Learning Representations, Banff, Apr 14-16, 2014.
[23] ZHONG S T, WANG C D, LAI J H, et al. An autoencoder framework with attention mechanism for cross-domain recommendation[J]. IEEE Transactions on Cybernetics, 2022, 52(6): 5229-5241.
[24] HAO X B, LIU Y D, XIE R B, et al. Adversarial feature translation for multi-domain recommendation[C]//Proceedings of the 27th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: ACM, 2021: 2964-2973.
[25] HE M, ZHANG J L, YANG P, et al. Robust transfer learning for cross-domain collaborative filtering using multiple rating patterns approximation[C]//Proceedings of the 11th ACM International Conference on Web Search and Data Mining. New York: ACM, 2018: 225-233.
[26] HU G N, ZHANG Y, YANG Q. CoNet: collaborative cross networks for cross-domain recommendation[C]//Proceedings of the 2018 ACM Conference on Information and Knowledge Management. New York: ACM, 2018: 667-676.
[27] GAO C, CHEN X N, FENG F L, et al. Cross-domain recommendation without sharing user-relevant data[C]//Proceed-ings of the 2019 World Wide Web Conference, San Francisco, May 13-17, 2019. New York: ACM, 2019: 491-502.
[28] LI P, TUZHILIN A. DDTCDR: deep dual transfer cross domain recommendation[C]//Proceedings of the 13th ACM International Conference on Web Search and Data Mining. New York: ACM, 2020: 331-339.
[29] ZHANG Y N, LIU Y, HAN P, et al. Learning personalized itemset mapping for cross-domain recommendation[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence, Yokohama, Jul 2020: 2561-2567.
[30] ZHAO C, LI C L, XIAO R, et al. CATN: cross-domain recommendation for cold-start users via aspect transfer network[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 229-238.
[31] MAN T, SHEN H W, JIN X L, et al. Cross-domain recommendation: an embedding and mapping approach[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Aug 19-25, 2017: 2464-2470.
[32] PAN W K, XIANG E W, LIU N N, et al. Transfer learning in collaborative filtering for sparsity reduction[C]//Proceedings of the 24th AAAI Conference on Artificial Intelligence.Menlo Park: AAAI, 2010: 230-235.
[33] FU W J, PENG Z H, WANG S Z, et al. Deeply fusing reviews and contents for cold start users in cross-domain recommendation systems[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Menlo Park: AAAI, 2019: 94-101.
[34] BACHMAN P, HJELM R D, BUCHWALTER W. Learning representations by maximizing mutual information across views[C]//Advances in Neural Information Processing Systems 32, Vancouver, Dec 8-14, 2019: 15509-15519.
[35] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. [2023-03-12]. https://arxiv.org/abs/1810.04805.
[36] ZHOU K, WANG H, ZHAO W X, et al. S3-Rec: self-supervised learning for sequential recommendation with mutual information maximization[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management. New York: ACM, 2020: 1893-1902.
[37] XIE X, SUN F, LIU Z, et al. Contrastive pre-training for sequential recommendation[EB/OL]. [2023-03-12]. https://arxiv.org/abs/2010.14395.
[38] 王永贵, 赵晓暄. 结合自监督学习的图神经网络会话推荐[J]. 计算机工程与应用, 2023, 59(3): 244-252.
WANG Y G, ZHAO X X. Self-supervised graph neural networks for session-based recommendation[J]. Computer Engineering and Applications, 2023, 59(3): 244-252.
[39] XIA X, YIN H, YU J, et al. Self-supervised hypergraph convolutional networks for session-based recommendation[C]//Proceedings of the 2021 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2021: 4503-4511.
[40] VELICKOVIC P, FEDUS W, HAMILTON W L, et al. Deep graph infomax[C]//Proceedings of the 7th International Conference on Learning Representations, New Orleans, May 6-9, 2019.
[41] 申翔翔, 侯新文, 尹传环. 深度强化学习中状态注意力机制的研究[J]. 智能系统学报, 2020, 15(2): 317-322.
SHEN X X, HOU X W, YIN C H. State attention in deep reinforcement learning[J]. CAAI Transactions on Intelligent Systems, 2020, 15(2): 317-322.
[42] 高芬, 苏依拉, 牛向华, 等. 基于Transformer的蒙汉神经机器翻译研究[J]. 计算机应用与软件, 2020, 37(2): 141-146.
GAO F, SU Y L, NIU X H, et al. Research on Mongolian Chinese neural machine translation based on Transformer[J]. Computer Applications and Software, 2020, 37(2): 141-146.
[43] CHAUDHARI S, POLATKAN G, RAMANATH R, et al. An attentive survey of attention[EB/OL]. [2023-03-12]. https://arxiv.org/pdf/1904.02874.pdf.
[44] 任欢, 王旭光. 注意力机制综述[J]. 计算机应用, 2021, 41(S1): 1-6.
REN H, WANG X G. Overview of attention mechanism[J].Journal of Computer Applications, 2021, 41(S1): 1-6.
[45] SONG W P, SHI C C, XIAO Z P, et al. AutoInt: automatic feature interaction learning via self-attentive neural networks[C]//Proceedings of the 28th ACM International Conference on Information and Knowledge Management, Beijing, Nov 3-7, 2019. Menlo Park: AAAI, 2019: 1161-1170.
[46] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Advances in Neural Information Processing Systems 30, Long Beach, Dec 4-9, 2017: 5998-6008.
[47] DENG Z H, HUANG L, WANG C D, et al. DeepCF: a unified framework of representation learning and matching function learning in recommender system[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Menlo Park: AAAI, 2019: 61-68.
[48] XI W D, HUANG L, WANG C D, et al. BPAM: recommendation based on BP neural network with attention mechanism[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 3905-3911.
[49] HU L, CAO J, XU G, et al. Personalized recommendation via cross-domain triadic factorization[C]//Proceedings of the 22nd International World Wide Web Conference, Rio de Janeiro, May 13-17, 2013. New York: ACM, 2013: 595-606.
[50] SAHU A K, DWIVEDI P. Matrix factorization in cross-domain recommendations framework by shared users latent factors[J]. Computer Journal, 2018, 143(1): 387-394.
[51] ELKAHKY A M, SONG Y, HE X. A multi-view deep learning approach for cross domain user modeling in recommendation systems[C]//Proceedings of the 24th International Conference on World Wide Web, Florence, May 18-22, 2015. New York: ACM, 2015: 278-288.
[52] YUAN F, YAO L, BENATALLAH B. DARec: deep domain adaptation for cross-domain recommendation via transferring rating patterns[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 4227-4233.