融合多视图对比学习的知识图谱补全算法

doi:10.3778/j.issn.1673-9418.2301038

摘要/Abstract

摘要： 知识图谱补全是基于知识图谱中已有的实体和关系，推理新的三元组的过程。现有的方法通常使用编码器-解码器框架，在编码器中使用图卷积神经网络将三元组中的实体和关系编码为嵌入向量，在解码器中根据实体关系的嵌入计算各个尾实体的评分，评分最高的尾实体作为推理结果。解码器部分都是独立地对三元组进行推理，很少考虑图级别的嵌入信息。因此提出了融合对比学习的图谱补全算法，在模型中加入了多视图对比学习，对图级别的嵌入信息进行了约束。模型中多个视图的互相对比为三元组关系构造了不同的分布空间，不同关系分布互相拟合，更适合补全任务的学习。对比学习对实体和子图的嵌入向量的约束，增强了模型的补全效果。在两个基准数据集上进行了实验，结果表明，在数据集FB15k-237中，MRR比方法A2N提高了12.6%，比InteractE提高了0.8%。在数据集WN18RR上，MRR比A2N提高了7.3%，比InteractE提高了4.3%。实验结果表明，该方法优于已有补全算法。

关键词: 知识图谱, 链接预测, 对比学习, 编码器, 解码器

Abstract: Knowledge graph completion is a process of reasoning new triples based on existing entities and relations in knowledge graph. The existing methods usually use the encoder-decoder framework. Encoder uses graph convolutional neural network to get the embeddings of entities and relations. Decoder calculates the score of each tail entity according to the embeddings of the entities and relations. The tail entity with the highest score is the inference result. Decoder inferences triples independently, without consideration of graph information. Therefore, this paper proposes a graph completion algorithm based on contrastive learning. This paper adds a multi-view contrastive learning framework into the model to constrain the embedded information at graph level. The comparison of multiple views in the model constructs different distribution spaces for relations. Different distributions of relations fit each other, which is more suitable for completion tasks. Contrastive learning constraints the embedding vectors of entity and subgraph and enhahces peroformance of the task. Experiments are carried out on two datasets. The results show that MRR is improved by 12.6% over method A2N and 0.8% over InteractE on FB15k-237 dataset, and 7.3% over A2N and 4.3% over InteractE on WN18RR dataset. Experimental results demonstrate that this model outperforms other completion methods.

Key words: knowledge graph, link prediction, contrastive learning, encoder, decoder

乔梓峰, 秦宏超, 胡晶晶, 李荣华, 王国仁. 融合多视图对比学习的知识图谱补全算法[J]. 计算机科学与探索, 2024, 18(4): 1001-1009.

QIAO Zifeng, QIN Hongchao, HU Jingjing, LI Ronghua, WANG Guoren. Knowledge Graph Completion Algorithm with Multi-view Contrastive Learning[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(4): 1001-1009.

参考文献

[1] CHOENMACKERS S, DAVIS J, ETZIONI O, et al. Learning first-order horn clauses from web text[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Oct 9-11, 2010. Stroudsburg: ACL, 2010: 1088-1098.
[2] GALáRRAGA L, TELIOUDI C, HOSE K, et al. Fast rule mining in ontological knowledge bases with AMIE+[J]. The International Journal on Very Large Data Bases, 2015, 24(6): 707-730.
[3] GALáRRAGA L, TELIOUDI C, HOSE K, et al. AMIE: association rule mining under incomplete evidence in ontological knowledge bases[C]//Proceedings of the 22nd International Conference on World Wide Web, Rio de Janeiro, May 13-17, 2013. New York: ACM, 2013: 413-422.
[4] KOK S, DOMINGOS P. Learning the structure of Markov logic networks[C]//Proceedings of the 22nd International Conference on Machine Learning, Bonn, Aug 7-11, 2005. New York: ACM, 2005: 441-448.
[5] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Trans-lating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-10, 2013. Red Hook: Curran Associates Inc., 2013: 2787-2795．
[6] NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning, Bellevue, Jun 28-Jul 2, 2011. Madison: OmniPress, 2011: 809-816.
[7] YANG B, YIH W T, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases[C]//Proceedings of the 2015 International Conference on Learning Representations, San Diego, May 7-9, 2015.
[8] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning, Jun 19-24, 2016: 2071-2080.
[9] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 1811-1818.
[10] DAI Q N, TU D N, NGUYEN D Q, et al. A novel embedding model for knowledge base completion based on convolutional neural network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 327-333.
[11] VASHISHTH S, SANYAL S, NITIN V, et al. InteractE: improving convolution-based knowledge graph embeddings by increasing feature interactions[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 3009-3016.
[12] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the 15th International Conference on the Semantic Web, Heraklion, Jun 3-7, 2018. Cham: Springer, 2018: 593-607.
[13] NATHANI D, CHAUHAN J, SHARMA C, et al. Learning attention based embeddings for relation prediction in knowledge graphs[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 4710-4723.
[14] VASHISHTH S, SANYAL S, NITIN N, et al. Composition-based multi-relational graph convolutional networks[C]//Proceedings of the 8th International Conference on Learning Representations, Addis Ababa, Apr 26-30, 2020.
[15] JAISWAL A, BABU A R, ZADEH M Z, et al. A survey on contrastive self-supervised learning[J]. Technologies, 2020, 9(1): 2．
[16] VELIC KOVIC P, FEDUS W, HAMILTON W L, et al. Deep graph infomax[C]//Proceeding of the 2019 International Conference on Learning Representations, New Orleans, May 6-9, 2019.
[17] YOU Y N, CHEN T L, SUI Y D, et al. Graph contrastive learning with augmentations[C]//Advances in Neural Information Processing Systems 33, Vancouver, Dec 6-12, 2020. Red Hook: Curran Associates Inc., 2020: 5812-5823.
[18] HASSANI K, AHMADI A H K. Contrastive multi-view representation learning on graphs[C]//Proceedings of the 37th International Conference on Machine Learning, Jul 13-18, 2020: 4116-4126.
[19] DUVENAUD D K, MACLAURIN D, IPARRAGUIRRE J, et al. Convolutional networks on graphs for learning molecular fingerprints[C]//Advances in Neural Information Processing Systems 28, Montreal, Dec 7-12, 2015. Cambridge:MIT Press, 2015: 2224-2232.
[20] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, Aug 6-11, 2017: 1263-1272.
[21] ATWOOD J, TOWSLEY D. Diffusion-convolutional neural networks[C]//Advances in Neural Information Processing Systems 29, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates Inc., 2016: 2001-2009.
[22] SIMONOVSKY M, KOMODAKIS N. Dynamic edge-conditioned filters in convolutional neural networks on graphs[C]//Proceedings of the 2017 Conference on Computer Vision and Pattern Recognition, Honolulu, Jul 21-26, 2017. Washington: IEEE Computer Society, 2017: 29-38.
[23] SOHN K. Improved deep metric learning with multi-class n-pair loss objective[C]//Advances?in?Neural?Information?Processing?Systems?29, Barcelona, Dec 5-10, 2016. Red Hook: Curran Associates Inc., 2016: 1857-1865.
[24] WANG F, LIU H. Understanding the behaviour of contrastive loss[C]//Proceeding of the 2021 IEEE Conference on Computer Vision and Pattern Recognition, Jun 19-25, 2021. Washington: IEEE Computer Society, 2021: 2495-2504．
[25] TOUTANOVA K,CHEN D Q, PANTEL P, et al. Representing text for joint embedding of text and knowledge bases[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Stroudsburg: ACL, 2015: 1499-1509.
[26] BOLLACKER K D, EVANS C, PARITOSH P K, et al. FreeBase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York: ACM，2008: 1247-1250.
[27] BALAZEVIC I, ALLEN C, TIMOTHY M. Hospedales: multi-relational Poincaré graph embeddings[C]//Advances in Neural Information Processing Systems 32, Vancouver, Dec 8-14, 2019. Red Hook: Curran Associates Inc., 2019: 4465-4475.
[28] BANSAL T, JUAN D C, RAVI S，et al. A2N: attending to neighbors for knowledge graph inference[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 4387-4392.
[29] ZHANG Z Q, WANG J, YE J P, et al. Rethinking graph convolutional networks in knowledge graph completion[C]//Proceedings of the ACM Web Conference 2022, Lyon, Apr 25-29, 2022. New York: ACM, 2022: 798-807.
[30] NICKEL M, ROSASCO L, POGGIO T. Holographic embeddings of knowledge graphs[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Feb 12-17, 2016. Menlo Park: AAAI, 2016: 1955-1961.