Contrast Research of Representation Learning in Entity Alignment Based on Graph Neural Network

doi:10.3778/j.issn.1673-9418.2307053

Abstract

Abstract: Entity alignment is an important step in knowledge fusion, which aims to identify equivalent entities in different knowledge graphs. In order to accurately determine the equivalent entities, the existing methods first perform representation learning to map the entities into a low-dimensional vector space, and then infer the equivalence of the entities by the similarity between the vectors. Recent works on entity alignment focus on the improvement of representation learning methods. In order to better understand the mechanism of these models, mine valuable design directions, and provide reference for subsequent optimization and improvement work, this paper reviews the research on representation learning methods for entity alignment. Firstly, based on the existing methods, a general framework for representation learning is proposed, and several representative works are summarized and analyzed. Then, these works are compared and analyzed through experiments, and the common methods of each module in the framework are compared. Through the results, the advantages and disadvantages of various methods are summarized, and the use suggestions are put forward. Finally, the feasibility of the alignment and fusion of large language models and knowledge graphs is preliminarily discussed, and the existing problems and challenges are analyzed.

Key words: knowledge fusion, entity alignment, representation learning, graph neural network, large language model

摘要： 实体对齐是知识融合的一个重要步骤，其目的在于识别不同知识图谱中的等价实体。为准确判断出对等的实体，现有方法首先进行表示学习，将实体映射到低维向量空间中，接着通过向量间的相似度推断实体的等价性。而近期实体对齐的相关工作也大都聚焦于表示学习方法的改进上。为了能够更好地理解这些模型的机理，挖掘有价值的设计思路，并为后续的优化改进工作提供参考，对实体对齐表示学习方法进行了研究综述。首先基于现有方法，提出了一个通用的表示学习框架，并用该框架对几个具有代表性的工作进行了归纳概括以及分析解构。接着通过实验对这些工作进行了对比分析，并对框架中各个模块的常见方法进行了比较。根据实验结果，总结了各种方法的优劣，并提出了使用建议。最后初步讨论了大规模语言模型与知识图谱对齐融合的可行性，并分析了存在的问题以及潜在的挑战。

关键词: 知识融合, 实体对齐, 表示学习, 图神经网络, 语言大模型

PENG Huang, ZENG Weixin, ZHOU Jie, TANG Jiuyang, ZHAO Xiang. Contrast Research of Representation Learning in Entity Alignment Based on Graph Neural Network[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(10): 2343-2357.

彭鐄, 曾维新, 周杰, 唐九阳, 赵翔. 基于图神经网络的实体对齐表示学习方法比较研究[J]. 计算机科学与探索, 2023, 17(10): 2343-2357.

References

[1] EHRLINGER L, W?? W. Towards a definition of knowledge graphs[C]//Proceedings of the Posters and Demos Track of the 12th International Conference on Semantic Systems, Leipzig, Sep 12-15, 2016, 1695: 1-4.
[2] WANG X, HE X N, CAO Y X, et al. KGAT: knowledge graph attention network for recommendation[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, Anchorage, Aug 4-8, 2019. New York: ACM, 2019: 950-958.
[3] CHEN C, ZHANG M, MA W Z, et al. Jointly non-sampling learning for knowledge graph enhanced recommendation[C]//Proceedings of the 43rd International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul 25-30, 2020. New York: ACM, 2020: 189-198.
[4] HU S, ZOU L, YU J X, et al. Answering natural language questions by subgraph matching over knowledge graphs[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 30(5): 824-837.
[5] CUI W, XIAO Y, WANG H, et al. KBQA: learning question answering over QA corpora and knowledge bases[J]. Proceedings of the VLDB Endowment, 2016, 10(5): 565-576.
[6] PUJARA J, MIAO H, GETOOR L, et al. Knowledge graph identification[C]//Proceedings of the 12th International Semantic Web Conference, Sydney, Oct 21-25, 2013. Berlin, Heidelberg: Springer, 2013: 542-557.
[7] 赵晓娟, 贾焰, 李爱平,等. 多源知识融合技术研究综述[J]. 云南大学学报(自然科学版), 2020, 42(3): 459-473.
ZHAO X J, JIA Y, LI A P, et al. A survey of the research on multi-source knowledge fusion technology[J]. Journal of Yunnan University (Natural Sciences Edition), 2020, 42(3): 459-473.
[8] SUN Z, ZHANG Q, HU W, et al. A benchmarking study of embedding-based entity alignment for knowledge graphs[J]. Proceedings of the VLDB Endowment, 2020, 13(12): 2326-2340.
[9] ZENG W X, ZHAO X, TAN Z, et al. Matching knowledge graphs in entity embedding spaces: an experimental study[J]. IEEE Transactions on Knowledge and Data Engineering, 2023. DOI: 10.1109/TKDE.2023.3272584.
[10] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems 26, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 2787-2795.
[11] WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, Quebec City, Jul 27-31, 2014. Palo Alto: AAAI, 2014: 1112-1119.
[12] SUN Z, HU W, ZHANG Q, et al. Bootstrapping entity alignment with knowledge graph embedding[C]//Proceedings of the 27th International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018. San Francisco: Margan Kaufmann, 2018: 4396-4402.
[13] CHEN M H, TIAN Y T, YANG M H, et al. Multilingual knowledge graph embeddings for cross-lingual knowledge alignment[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Aug 19-25, 2017. San Francisco: Margan Kaufmann, 2017: 1511-1517.
[14] KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[C]//Proceedings of the 5th International Conference on Learning Representations, Toulon, Apr 24-26, 2017: 1-14.
[15] GILMER J, SCHOENHOLZ S S, RILEY P F, et al. Neural message passing for quantum chemistry[C]//Proceedings of the 34th International Conference on Machine Learning, Sydney, Aug 6-11, 2017: 1263-1272.
[16] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the 15th International Conference on Semantic Web, Heraklion, Jun 3-7, 2018. Cham: Springer, 2018: 593-607.
[17] ZHU Q N, ZHOU X F, WU J, et al. Neighborhood-aware attentional representation for multilingual knowledge graphs[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019. San Francisco: Margan Kaufmann, 2019: 1943-1949.
[18] YANG H W, ZOU Y Y, SHI P, et al. Aligning cross-lingual entities with multi-aspect information[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 4431-4441.
[19] WU Y T, LIU X, FENG Y S, et al. Relation-aware entity alignment for heterogeneous knowledge graphs[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019. San Francisco: Margan Kaufmann, 2019: 5278-5284.
[20] CAO Y, LIU Z, LI C, et al. Multi-channel graph neural network for entity alignment[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 1452-1461.
[21] VELI?KOVI? P, CUCURULL G, CASANOVA A, et al. Graph attention networks[C]//Proceedings of the 6th International Conference on Learning Representations, Vancouver, Apr 30-May 3, 2018: 1-12.
[22] WANG Z C, LV Q S, LAN X H, et al. Cross-lingual knowledge graph alignment via graph convolutional networks[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 349-357.
[23] XU K, WANG L, YU M, et al. Cross-lingual knowledge graph alignment via graph matching neural network[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 3156-3161.
[24] SUN Z Q, WANG C M, HU W, et al. Knowledge graph alignment network with gated multi-hop neighborhood aggregation[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, New York, Feb 7-12, 2020. Palo Alto: AAAI, 2020: 222-229.
[25] MAO X, WANG W T, XU H M, et al. MRAEA: an efficient and robust entity alignment approach for cross-lingual knowledge graph[C]//Proceedings of the 13th International Conference on Web Search and Data Mining, Houston, Feb 3-7, 2020. New York: ACM, 2020: 420-428.
[26] MAO X, WANG W T, XU H M, et al. Relational reflection entity alignment[C]//Proceedings of the 29th ACM International Conference on Information & Knowledge Management, Ireland, Oct 19-23, 2020. New York: ACM, 2020: 1095-1104.
[27] CAI W, MA W, ZHAN J, et al. Entity alignment with reliable path reasoning and relation-aware heterogeneous graph transformer[C]//Proceedings of the 31st International Joint Conferences on Artificial Intelligence Organization, Vienna, Jul 23-29, 2022. Messe Wien: IJCAI, 2022: 1930-1937.
[28] WU Y T, LIU X, FENG Y S, et al. Neighborhood matching network for entity alignment[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 6477-6487.
[29] ZHU R, MA M, WANG P. RAGA: relation-aware graph attention networks for global entity alignment[C]//Proceedings of the 25th Pacific-Asia Conference on Knowledge Discovery and Data Mining, May 11-14, 2021. Cham: Springer, 2021: 501-513.
[30] LIU Z Y, CAO Y X, PAN L M, et al. Exploring and evaluating attributes, values, and structures for entity alignment[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 6355-6364.
[31] MAO X, WANG W T, WU Y B, et al. Are negative samples necessary in entity alignment? An approach with high performance, scalability and robustness[C]//Proceedings of the 30th ACM International Conference on Information & Know-ledge Management, Queensland, Nov 1-5, 2021. New York: ACM, 2021: 1263-1273.
[32] MAO X, WANG W T, WU Y B, et al. Boosting the speed of entity alignment 10×: dual attention matching network with normalized hard sample mining[C]//Proceedings of the Web Conference 2021, Ljubljana, Apr 19-23, 2021. New York: ACM, 2021: 821-832.
[33] ZHONG Z, ZHANG M, FAN J, et al. Semantics driven embedding learning for effective entity alignment[C]//Proceedings of the 2022 IEEE 38th International Conference on Data Engineering, Kuala Lumpur, May 9-12, 2022. Piscataway: IEEE, 2022: 2127-2140.
[34] SRIVASTAVA R K, GREFF K, SCHMIDHUBER J. Highway networks[J]. arXiv:1505.00387, 2015.
[35] SUN Y F, CHENG C M, ZHANG Y H, et al. Circle loss: a unified perspective of pair similarity optimization[C]//Proceedings of the 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, Jun 13-19, 2020. Piscataway: IEEE, 2020: 6398-6407.
[36] KOTNIS B, NASTASE V. Analysis of the impact of negative sampling on link prediction in knowledge graphs[J]. arXiv:1708.06816, 2017.
[37] HE K M, ZHANG X Y, REN S Q, et al. Deep residual learning for image recognition[C]//Proceedings of the 2016 IEEE Conference on Computer Vision and Pattern Recognition, Las Vegas, Jun 27-30, 2016. Washington: IEEE Computer Society, 2016: 770-778.
[38] SUN Z, HU W, LI C. Cross-lingual entity alignment via joint attribute-preserving embedding[C]//Proceedings of the 16th International Semantic Web Conference, Vienna, Oct 21-25, 2017. Cham: Springer, 2017: 628-644.
[39] PENNINGTON J, SOCHER R, MANNING C D. GloVe: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1532-1543.
[40] LI Y J, TARLOW D, BROCKSCHMIDT M, et al. Gated graph sequence neural networks[C]//Proceedings of the 4th International Conference on Learning Representations, San Juan, May 2-4, 2016: 1-20.
[41] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 4171-4186.
[42] HAMILTON W L, YING Z T, LESKOVEC J. Inductive representation learning on large graphs[C]//Advances in Neural Information Processing Systems 30, Long Beach, Dec 4-9, 2017: 1024-1034.
[43] CLEVERT D A, UNTERTHINER T, HOCHREITER S. Fast and accurate deep network learning by exponential linear units (ELUs)[C]//Proceedings of the 4th International Conference on Learning Representations, San Juan, May 2-4, 2016: 1-14.
[44] GRILL J B, STRUB F, ALTCHé F, et al. Bootstrap your own latent-a new approach to self-supervised learning[C]//Advances in Neural Information Processing Systems 33, Dec 6-12, 2020: 21271-21284.
[45] CHEN X L, HE K M. Exploring simple siamese representation learning[C]//Proceedings of the 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, Jun 19-25, 2021. Piscataway: IEEE, 2021: 15750-15758.
[46] HE K M, ZHANG X Y, REN S Q, et al. Delving deep into rectifiers: surpassing human-level performance on Image-Net classification[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision, Santiago, Dec 7-13, 2015. Washington: IEEE Computer Society, 2015: 1026-1034.
[47] IOFFE S, SZEGEDY C. Batch normalization: accelerating deep network training by reducing internal covariate shift[C]//Proceedings of the 32nd International Conference on Machine Learning, Lille, Jul 6-11, 2015. Cambridge: MIT Press, 2015: 448-456.
[48] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1724-1734.
[49] ZENG K S, DONG Z H, HOU L, et al. Interactive contrastive learning for self-supervised entity alignment[C]//Proceedings of the 31st ACM International Conference on Information & Knowledge Management, Atlanta, Oct 17-21, 2022. New York: ACM, 2022: 2465-2475.
[50] LIU X, HONG H, WANG X, et al. SelfKG: self-supervised entity alignment in knowledge graphs[C]//Proceedings of the ACM Web Conference 2022, Lyon, Apr 25-29, 2022. New York: ACM, 2022: 860-870.
[51] WANG C G, LIU X, SONG D. Language models are open knowledge graphs[J]. arXiv:2010.11967, 2020.
[52] PETRONI F, ROCKT?SCHEL T, RIEDEL S, et al. Language models as knowledge bases?[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 2463-2473.
[53] ZHONG Z X, FRIEDMAN D, CHEN D Q. Factual probing is [MASK]: learning vs. learning to recall[C]//Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Jun 6-11, 2021. Stroudsburg: ACL, 2021: 5017-5033.