联合注意力与卷积网络的知识超图链接预测

doi:10.3778/j.issn.1673-9418.2208071

摘要/Abstract

摘要： 知识超图（KHG）是超图结构的知识图谱。知识超图链接预测旨在通过已知实体和关系预测缺失的关系。然而，现有最优的基于嵌入模型的知识超图链接预测方法HypE虽然实体嵌入时考虑了位置信息，但关系嵌入时忽略了不同实体的贡献有差异，且实体嵌入蕴含信息不够充足。关系嵌入考虑实体贡献度并补足实体嵌入的信息含量，可以较大地提升模型的预测能力。因此，提出了一种基于注意力与卷积网络的链接预测方法（LPACN），采用改进的注意力机制将实体的注意力信息融入到关系嵌入中；并且将同元组内相邻实体个数信息融入卷积网络，进一步补足了实体卷积向量的信息含量。针对LPACN的梯度消失问题，将改进的残差网络（ResidualNet）融入LPACN，并引入多层感知器（MLP）提升了模型的非线性学习能力，得到LPACN的改进算法LPACN+。真实数据集上的大量实验验证了LPACN的预测性能均优于Baseline方法。

关键词: 知识超图, 链接预测, 嵌入模型, 注意力机制

Abstract: Knowledge hypergraphs (KHG) are knowledge graph of hypergraph structure. KHG link prediction aims to predict the missing relations through the known entities and relations. However, HypE, the existing optimal KHG link prediction method based on embedding model considers the location information when embedding entities, but ignores the differences in the contributions of different entities when embedding relations. And the information of the entity convolution vector is insufficient. Relation embeddings consider the entity contribution and supply the information of entity embedding, which can greatly improve the prediction ability of model. Therefore, link prediction based on attention and convolution network (LPACN) is proposed. The improved attention mechanism is applied to merging entity attention information into relation embeddings. And the number information of neighboring entities in the same tuple is integrated into the convolution network, which further supplies the information of entity convolution embedding. For the gradient vanishing problem of LPACN, the improved ResidualNet is integrated into LPACN, and the multilayer perceptron (MLP) is used to improve the nonlinear learning ability of model. The improved algorithm LPACN+ is obtained. Extensive experiments on real datasets validate that LPACN is better than the Baselines.

Key words: knowledge hypergraph, link prediction, embedding model, attention mechanism

庞俊, 徐浩, 秦宏超, 林晓丽, 刘小琪, 王国仁. 联合注意力与卷积网络的知识超图链接预测[J]. 计算机科学与探索, 2023, 17(11): 2734-2742.

PANG Jun, XU Hao, QIN Hongchao, LIN Xiaoli, LIU Xiaoqi, WANG Guoren. Link Prediction in Knowledge Hypergraph Combining Attention and Convolution Network[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(11): 2734-2742.

参考文献

[1] 张大坤, 任淑霞. 超图可视化方法研究综述[J]. 计算机科学与探索, 2018, 12(11): 1701-1717.
ZHANG D K, REN S X. Survey on hypergraph visualization method[J]. Journal of Frontiers of Computer Science and Technology, 2018, 12(11): 1701-1717.
[2] FATEMI B, TASLAKIAN P, VAZQUEZ D, et al. Knowledge hypergraphs: prediction beyond binary relations[C]//Proceedings of the 29th International Joint Conference on Artificial Intelligence, Yokohama, 2020: 2191-2197.
[3] 武家伟, 孙艳春. 融合知识图谱和深度学习方法的问诊推荐系统[J]. 计算机科学与探索, 2021, 15(8): 1432-1440.
WU J W, SUN Y C. Recommendation system for medical consultation integrating knowledge graph and deep learning methods[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(8): 1432-1440.
[4] 孙亚伟, 程龚, 厉肖, 等. 基于图匹配网络的可解释知识图谱复杂问答方法[J]. 计算机研究与发展, 2021, 58(12): 2673-2683.
SUN Y W, CHENG G, LI X, et al. Graph matching network for interpretable complex question answering over knowledge graphs[J]. Journal of Computer Research and Development, 2021, 58(12): 2673-2683.
[5] KAZEMI S M, BUCHMAN D, KERSTING K, et al. Relational logistic regression[C]//Proceedings of the 14th International Conference on Principles of Knowledge Representation and Reasoning, Vienna, Jul 20-24, 2014. Menlo Park: AAAI, 2014: 1-10.
[6] GUAN S P, JIN X L, WANG Y Z, et al. Link prediction on n-ary relational data[C]//Proceedings of the 2019 World Wide Web Conference, San Francisco, May 13-17, 2019. New York: ACM, 2019: 583-593.
[7] GUAN S P, JIN X L, GUO J F, et al. Neuinfer: knowledge inference on n-ary facts[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 6141-6151.
[8] ROSSO P, YANG D Q, CUDRé-MAUROUX P. Beyond triplets: hyper-relational knowledge graph embedding for link prediction[C]//Proceedings of the Web Conference 2020, Taipei, China, Apr 20-24, 2020. New York: ACM, 2020: 1885-1896.
[9] WEN J F, LI J X, MAO Y Y, et al. On the representation and embedding of knowledge bases beyond binary relations[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Menlo Park: AAAI, 2016: 1300-1307.
[10] LIU Y, YAO Q M, LI Y. Generalizing tensor decomposition for n-ary relational knowledge bases[C]//Proceedings of the Web Conference 2020, Taipei, China, Apr 20-24, 2020. New York: ACM, 2020: 1104-1114.
[11] 郭正山, 左劼, 段磊, 等. 面向知识超图链接预测的生成对抗负采样方法[J]. 计算机研究与发展, 2022, 59(8): 1742-1756.
GUO Z S, ZUO J, DUAN L, et al. A generative adversarial negative sampling method for knowledge hypergraph link prediction[J]. Journal of Computer Research and Development, 2022, 59(8): 1742-1756.
[12] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2013, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 2787-2795.
[13] WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec City, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 1112-1119.
[14] BALAZEVIC I, ALLEN C, HOSPEDALES T. TuckER: tensor factorization for knowledge graph completion[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5184-5193.
[15] KAZEMI S M, POOLE D. Simple embedding for link prediction in knowledge graphs[C]//Proceedings of the Annual Conference on Neural Information Processing Systems 2018, Montréal, Dec 3-8, 2018: 4289-4300.
[16] NATHANI D, CHAUHAN J, SHARMA C, et al. Learning attention-based embeddings for relation prediction in know-ledge graphs[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 4710-4723.
[17] KADLEC R, BAJGAR O, KLEINDIENST J. Knowledge base completion: baselines strike back[C]//Proceedings of the 2nd Workshop on Representation Learning for NLP, Vancouver, Aug 3, 2017. Stroudsburg: ACL, 2017: 69-74.
[18] HE K, ZHANG X, REN S, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[19] LI R Y M, TANG B, CHAU K W. Sustainable construction safety knowledge sharing: a partial least square-structural equation modeling and a feedforward neural network approach[J]. Sustainability, 2019, 11(20): 5831.
[20] NGUYEN D Q. An overview of embedding models of entities and relationships for knowledge base completion[J]. arXiv:1703.08098, 2017.
[21] MIKHAIL G, PRIYANSH T, GAURAV M, et al. Message passing for hyper-relational knowledge graphs[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 7346-7359.