Hierarchical Knowledge Graph Embedding and Self-Attention Mechanism for Recommendation

doi:10.3778/j.issn.1673-9418.2407019

Abstract

Abstract: Knowledge graphs have been widely applied in recommendation systems, contributing to alleviating the data sparsity issue in user-item interactions. Existing knowledge graph-based recommendation methods primarily rely on path mining or information propagation to explore potential associations between users and items. However, they often fail to fully leverage the rich semantics and structural information within knowledge graphs, and may introduce irrelevant noise, thereby affecting the accuracy of recommendations. To address these problems, this paper proposes a recommendation method that integrates hierarchical knowledge graph embedding and self-attention mechanism (HKSAR), aiming to extract high-order semantics and structural information from knowledge graphs to mitigate the sparsity problem. Specifically, the proposed model first constructs high-order subgraphs for user-item pairs, explicitly depicting the complex relationships between them. Through a hierarchical attention embedding learning process, the model encodes the high-order semantics and topological structure within the subgraphs, and employs a self-attention mechanism to differentiate the importance of each entity in the subgraphs, ultimately generating high-quality subgraph embeddings for accurate user preference modeling. Experimental results on three real-world datasets show that the proposed method achieves an average improvement of 10.7% and 13.6% in Hit and NDCG metrics, respectively, compared with the state-of-the-art baseline model. Moreover, the proposed method consistently outperforms in scenarios with varying degrees of data sparsity, effectively alleviating the data sparsity issue.

Key words: recommendation systems, graph neural networks, knowledge graphs, self-attention mechanism, collaborative filtering

摘要： 知识图谱在推荐系统中已广泛应用，有助于缓解用户与项目交互数据稀疏性问题。现有基于知识图谱的推荐方法主要依赖路径挖掘或信息传播来探索用户与项目之间的潜在关联，但未能充分利用知识图谱中的丰富语义和结构信息，且易引入无关噪声，影响推荐的准确性。为此，提出了一种融合层级知识图谱嵌入与注意力机制的推荐方法（HKSAR），旨在挖掘知识图谱的高阶语义和结构信息以缓解数据稀疏性问题。构建用户与项目的高阶子图，显式刻画用户与项目之间的复杂关联关系。通过层级子图嵌入模块，对子图中的高阶语义和拓扑结构进行编码，并结合自注意力机制，区分子图中各实体的重要性，最终生成高质量的子图嵌入表示，实现对用户偏好的精准建模。实验结果表明，在三个真实数据集上，相较于现有最优基线模型，所提方法在Hit和NDCG指标上分别实现了平均10.7%和13.6%的提升，且在不同数据稀疏程度的场景下推荐效果均占优，有效缓解了数据稀疏性问题。

SHA Xiao, WANG Jianwen, DING Jianchuan, XU Xiaoran. Hierarchical Knowledge Graph Embedding and Self-Attention Mechanism for Recommendation[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(6): 1508-1521.

沙潇, 王建文, 丁建川, 徐笑然. 融合层级知识图谱嵌入与注意力机制的推荐方法[J]. 计算机科学与探索, 2025, 19(6): 1508-1521.

References

[1] 温雯, 邓峰颖, 郝志峰, 等. 时空邻域感知的时序兴趣点推荐[J]. 计算机科学与探索, 2024, 18(7): 1865-1878.
WEN W, DENG F Y, HAO Z F, et al. Recommendation method for time-sequence point of interest via spatio-temporal vicinity perception[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(7): 1865-1878.
[2] WANG X, HE X N, WANG M, et al. Neural graph collaborative filtering[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2019: 165-174.
[3] 王永贵, 刘丹妮. 融合多个性化桥和自监督学习的跨域推荐算法[J]. 计算机科学与探索, 2024, 18(7): 1792-1805.
WANG Y G, LIU D N. Cross-domain recommendation algorithm combining multi-personalized bridges and self-supervised learning[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(7): 1792-1805.
[4] 刘源, 董永权, 陈成, 等. 融合热点与长短期兴趣的图神经网络课程推荐模型[J]. 计算机科学与探索, 2024, 18(6): 1600-1612.
LIU Y, DONG Y Q, CHEN C, et al. Graph neural network integrating hot spots and long and short-term interests for course recommendation[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(6): 1600-1612.
[5] WANG X, WANG D X, XU C R, et al. Explainable reasoning over knowledge graphs for recommendation[J]. Proceedings of the AAAI Conference on Artificial Intelligence, 2019, 33(1): 5329-5336.
[6] WANG X, HE X, CAO Y, et al. KGAT: knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2019:950-958.
[7] CAO Y X, WANG X, HE X N, et al. Unifying knowledge graph learning and recommendation: towards a better under-standing of user preferences[C]//Proceedings of the 2019 World Wide Web Conference. New York: ACM, 2019: 151-161.
[8] XIN X, HE X N, ZHANG Y F, et al. Relational collaborative filtering: modeling multiple item relations for recommendation[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2019: 125-134.
[9] CHEN C, ZHANG M, MA W Z, et al. Jointly non-sampling learning for knowledge graph enhanced recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 189-198.
[10] WANG P F, FAN Y, XIA L, et al. KERL: a knowledge-guided reinforcement learning model for sequential recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 209-218.
[11] MENG W J, YANG D Q, XIAO Y H. Incorporating user micro-behaviors and item knowledge into multi-task learning for session-based recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 1091-1100.
[12] WANG C Y, ZHANG M, MA W Z, et al. Make it a chorus: knowledge- and time-aware item modeling for sequential recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 109-118.
[13] YU X, REN X, SUN Y Z, et al. Personalized entity recommendation: a heterogeneous information network approach[C]//Proceedings of the 7th ACM International Conference on Web Search and Data Mining. New York: ACM, 2014: 283-292.
[14] SHI C, LIU J, ZHUANG F Z, et al. Integrating heterogeneous information via flexible regularization framework for recommendation[J]. Knowledge and Information Systems, 2016, 49(3): 835-859.
[15] SHI C, HU B B, ZHAO W X, et al. Heterogeneous information network embedding for recommendation[J]. IEEE Tran-sactions on Knowledge and Data Engineering, 2019, 31(2): 357-370.
[16] FU T Y, LEE W C, LEI Z. HIN2Vec: explore meta-paths in heterogeneous information networks for representation learning[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management. New York: ACM, 2017: 1797-1806.
[17] WANG X, JI H Y, SHI C, et al. Heterogeneous graph attention network[C]//Proceedings of the 2019 World Wide Web Conference. New York: ACM, 2019: 2022-2032.
[18] LU Y F, FANG Y, SHI C. Meta-learning on heterogeneous information networks for cold-start recommendation[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 1563-1573.
[19] SUN Z, YANG J, ZHANG J, et al. Recurrent knowledge graph embedding for effective recommendation[C]//Proceedings of the 12th ACM Conference on Recommender Systems. New York: ACM, 2018: 297-305.
[20] ZHAO K Z, WANG X T, ZHANG Y R, et al. Leveraging demonstrations for reinforcement recommendation reasoning over knowledge graphs[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 239-248.
[21] WANG X, XU Y K, HE X N, et al. Reinforced negative sampling over knowledge graph for recommendation[C]//Proceedings of the Web Conference 2020. New York: ACM, 2020: 99-109.
[22] WANG H W, ZHANG F Z, ZHANG M D, et al. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2019: 968-977.
[23] HAMILTON W L, YING R, LESKOVEC J. Inductive representation learning on large graphs[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2017: 1025-1035.
[24] CHEN X C, HUANG C R, YAO L N, et al. Knowledge-guided deep reinforcement learning for interactive recommendation[C]//Proceedings of the 2020 International Joint Conference on Neural Networks. Piscataway: IEEE, 2020: 1-8.
[25] ZHOU S J, DAI X Y, CHEN H K, et al. Interactive recommender system via knowledge graph-enhanced reinforcement learning[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 179-188.
[26] JIN J R, QIN J R, FANG Y C, et al. An efficient neighborhood-based interaction model for recommendation on heterogeneous graph[C]//Proceedings of the 26th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2020: 75-84.
[27] CHEN Y K, YANG Y M, WANG Y J, et al. Attentive knowledge-aware graph convolutional networks with collaborative guidance for personalized recommendation[C]//Proceedings of the 2022 IEEE 38th International Conference on Data Engineering. Piscataway: IEEE, 2022: 299-311.
[28] ZHOU R, HANSEN E A. Breadth-first heuristic search[J]. Artificial Intelligence, 2006, 170(4/5): 385-408.
[29] TARJAN R. Depth-first search and linear graph algorithms[C]//Proceedings of the 12th Annual Symposium on Switching and Automata Theory. Piscataway: IEEE, 1971: 114-121.
[30] PEROZZI B, AL-RFOU R, SKIENA S. DeepWalk: online learning of social representations[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2014: 701-710.
[31] HE X N, LIAO L Z, ZHANG H W, et al. Neural collaborative filtering[C]//Proceedings of the 26th International Conference on World Wide Web. New York: ACM, 2017: 173-182.
[32] KINGMA D, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations, 2015.
[33] WANG H, ZHANG F, WANG J, et al. RippleNet: propagating user preferences on the knowledge graph for recommender systems[C]//Proceedings of the 27th ACM International Conference on Information and Knowledge Management. New York: ACM, 2018: 417-426.
[34] HU B B, SHI C, ZHAO W X, et al. Leveraging meta-path based context for top-N recommendation with a neural co-attention model[C]//Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2018: 1531-1540.
[35] RENDLE S, FREUDENTHALER C, GANTNER Z, et al. BPR: Bayesian personalized ranking from implicit feedback[C]//Proceedings of the 25th Conference on Uncertainty in Artificial Intelligence. Arlington: AUAI Press, 2009: 452-461.
[36] HE X N, DENG K, WANG X, et al. LightGCN: simplifying and powering graph convolution network for recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2020: 639-648.
[37] ZHANG F Z, YUAN N J, LIAN D F, et al. Collaborative knowledge base embedding for recommender systems[C]//Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2016: 353-362.
[38] ZHAO H, YAO Q, LI J, et al. Meta-graph based recommendation fusion over heterogeneous information networks[C]//Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. New York: ACM, 2017: 635-644.