基于层次保留的知识图谱嵌入链路预测方法

doi:10.3778/j.issn.1673-9418.2207090

摘要/Abstract

摘要： 知识图谱嵌入（KGE）是预测知识图谱（KGs）中缺失链接的重要工具，它将知识图谱中的实体和关系嵌入到连续低维空间中，并尽可能地保留原数据中隐含的各种信息。近年来，一些知识图谱嵌入方法利用极坐标系对知识图谱中普遍存在的语义层次结构进行建模，提升了链路预测任务的性能。然而，这些方法在建模关系时，使用了简单的标度变换并过度关注于实体的层次差，这在一定程度上限制了模型的拟合力。为了应对上述问题，提出了基于层次保留的知识图谱嵌入方法（RHKE），它在建模知识图谱中的关系时考虑了实体本身的层次。具体来说，提出了混合变换，它包含一个倍率项和一个偏差项，当实体层次较低或较高时，标度变换主要受偏差项或倍率项影响。此外，由于变换后模型丢失了实体原本的层次，RHKE使用层次修正项，它将头尾实体的原本层次用不同比例组合后作为关系的附加信息。在多个公开数据集上的实验结果显示，RHKE在链路预测上的性能优于现有的语义层次模型。

关键词: 知识图谱嵌入（KGE）, 链路预测, 语义层次, 极坐标系

Abstract: Knowledge graph embedding (KGE) is an important tool to predict the missing links of knowledge graphs (KGs). It embeds entities and relations of KGs into continuous low-dimensional space and preserves the potential information of the original data as much as possible. Recently, some KGE methods model the common semantic hierarchies of KGs by utilizing polar coordinate system and improve the performance in link prediction task. However, they use the simple transform function and only focus on the hierarchical differences of entities when modeling relations, which limits the performance of the model. To address this issue, this paper proposes reserved hierarchy-based knowledge graph embeddings (RHKE). It considers the hierarchy of entity itself when modeling the relation. To be specific, RHKE proposes the mixed transform function, which contains a proportion item and a bias item. The transform function will be mainly affected by the proportion item or bias item when the hierarchy of entity is high or low. In addition, since the model loses the hierarchy of entity itself after the mixed transform, RHKE uses the hierarchy correction item, which is an additional information for the relation by combining the original hierarchy of the head and tail entity with different proportions. Experiments on several public datasets show that RHKE outperforms existing semantic hierarchical models in link prediction task.

Key words: knowledge graph embedding (KGE), link prediction, semantic hierarchy, polar coordinates

钱付兰, 王文学, 郑文杰, 陈洁, 赵姝. 基于层次保留的知识图谱嵌入链路预测方法[J]. 计算机科学与探索, 2023, 17(9): 2174-2183.

QIAN Fulan, WANG Wenxue, ZHENG Wenjie, CHEN Jie, ZHAO Shu. Reserved Hierarchy-Based Knowledge Graph Embedding for Link Prediction[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(9): 2174-2183.

参考文献

[1] MILLER G A. WordNet: a lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[2] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
[3] MAHDISOLTANI F, BIEGA J, SUCHANEK F. YAGO3: a knowledge base from multilingual wikipedias[C]//Procee-dings of the 7th Biennial Conference on Innovative Data Systems Research, Asilomar, Jan 4-7, 2015: 1-11.
[4] CARLSON A, BETTERIDGE J, KISIEL B, et al. Toward an architecture for never-ending language learning[C]//Pro-ceedings of the 24th AAAI Conference on Artificial Intel-ligence, Atlanta, Jul 11-15, 2010. Menlo Park: AAAI, 2010: 1306-1313.
[5] BAST H, HAUSSMANN E. More accurate question answer-ing freebase[C]//Proceedings of the 24th ACM International Conference on Information and Knowledge Management，Melbourne, Oct 19-23, 2015. New York: ACM, 2015: 1431-1440.
[6] LIANG J, FENG S, XIE C, et al. Bootstrapping informa-tion extraction via conceptualization[C]//Proceedings of the 37th IEEE International Conference on Data Engineering，Chania, Apr 19-22, 2021. Piscataway: IEEE, 2021: 49-60.
[7] LEE D, OH B, SEO S, et al. News recommendation with topic-enriched knowledge graphs[C]//Proceedings of the 29th ACM International Conference on Information and Knowle-dge Management，Ireland, Oct 19-23, 2020. New York: ACM, 2020: 695-704.
[8] WEST R, GABRILOVICH E, MURPHY K, et al. Knowle-dge base completion via search-based question answering[C]//Proceedings of the 23rd International Conference on World Wide Web，Seoul, Apr 7-11, 2014. New York: ACM, 2014: 515-526.
[9] WANG M, QIU L, WANG X. A survey on knowledge graph embeddings for link prediction[J]. Symmetry, 2021, 13(3): 485.
[10] JI S, PAN S, CAMBRIA E, et al. A survey on knowledge graphs: representation, acquisition, and applications[J]. IEEE Transactions on Neural Networks and Learning Systems, 2021, 33(2): 494-514.
[11] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-8, 2013. Cambridge: MIT Press, 2013: 2787-2795.
[12] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their composi-tionality[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems，Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 3111-3119.
[13] YANG B, YIH S W, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of the 3rd International Conference on Learn-ing Representations, San Diego, May 7-9, 2015: 1-12.
[14] SUN Z, DENG Z H, NIE J Y, et al. RotatE: knowledge graph embedding by relational rotation in complex space[C]//Proceedings of the 7th International Conference on Learning Representations, New Orleans, May 6-9, 2019: 1-18.
[15] WANG S, WEI X, DOS SANTOS C N, et al. H2KGAT: hierarchical hyperbolic knowledge graph attention network[C]//Proceedings of the 2020 Conference on Empirical Met-hods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 4952-4962.
[16] LI M, WANG Y, ZHANG D, et al. Link prediction in know-ledge graphs: a hierarchy-constrained approach[J]. IEEE Transactions on Big Data, 2022, 8(3): 630-643.
[17] ZHANG Z, CAI J, ZHANG Y, et al. Learning hierarchy-aware knowledge graph embeddings for link prediction[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Symposium on Educational Advances in Artificial Intelligence, New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 3065-3072.
[18] PAN Z, WANG P. Hyperbolic hierarchy-aware knowledge graph embedding for link prediction[C]//Proceedings of the Findings of the Association for Computational Linguistics, Punta Cana, Nov 16-20, 2021. Stroudsburg: ACL, 2021: 2941-2948.
[19] WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence，Québec City, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 1112-1119.
[20] LIN Y, LIU Z, SUN M, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Procee-dings of the 29th AAAI Conference on Artificial Intelli-gence, Austin, Jan 25-30, 2015. Menlo Park: AAAI，2015: 2181-2187.
[21] JI G, HE S, XU L, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing of the Asian Federation of Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 687-696.
[22] BALAZEVIC I, ALLEN C, HOSPEDALES T. Multi-rela-tional poincaré graph embeddings[C]//Proceedings of the 33rd Annual Conference on Neural Information Processing Systems, Vancouver, Dec 8-14, 2019. Cambridge: MIT Press, 2019: 4465-4475.
[23] LU H, HU H, LIN X. DensE: an enhanced non-commuta-tive representation for knowledge graph embedding with adaptive semantic hierarchy[J]. Neurocomputing, 2022, 476: 115-125.
[24] ZEB A, HAQ A U, CHEN J, et al. Learning hyperbolic attention-based embeddings for link prediction in knowle-dge graphs[J]. Knowledge-Based Systems, 2021, 229: 107369.
[25] INOUE M, FORSTER C H, DOS SANTOS A C. Semantic hierarchy-based convolutional neural networks for image classification[C]//Proceedings of the 2020 International Joint Conference on Neural Networks, Glasgow, Jul 19-24, 2020. Piscataway: IEEE, 2020: 1-8.
[26] XU C, LI R. Relation embedding with dihedral group in knowledge graph[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Jul 28- Aug 2, 2019. Stroudsburg: ACL, 2019: 263-272.
[27] GAO C, SUN C, SHAN L, et al. Rotate3D: representing relations as rotations in three-dimensional space for knowle-dge graph embedding[C]//Proceedings of the 29th ACM International Conference on Information and Knowledge Management, Ireland, Oct 19-23, 2020. New York: ACM, 2020: 385-394.
[28] ZHANG S, TAY Y, YAO L, et al. Quaternion knowledge graph embeddings[C]//Proceedings of the 33rd Annual Con-ference on Neural Information Processing Systems, Dec 8-14, 2019. Cambridge: MIT Press, 2019: 2731-2741.
[29] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Pro-ceedings of the 32nd AAAI Conference on Artificial Intelli-gence, the 30th Innovative Applications of Artificial Intelli-gence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 1811-1818.
[30] TOUTANOVA K, CHEN D. Observed versus latent features for knowledge base and text inference[C]//Proceedings of the 3rd Workshop on Continuous Vector Space Models and Their Compositionality, Beijing, Jul 26-31, 2015. Strouds-burg: ACL, 2015: 57-66.
[31] TANG Y, HUANG J, WANG G, et al. Orthogonal relation transforms with graph context modeling for knowledge graph embedding[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 2713-2722.
[32] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning,New York, Jun 19-24, 2016: 2071-2080.
[33] CHAMI I, WOLF A, JUAN D C, et al. Low-dimensional hyperbolic knowledge graph embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computa-tional Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 6901-6914.