Multimodal Unsupervised Entity Alignment Approach with Progressive Strategies

doi:10.3778/j.issn.1673-9418.2310100

Abstract

Abstract: Although the current entity alignment method utilizes the structural information between entities in the knowledge graph and achieves a good alignment effect, a large amount of side information contained between entities is ignored. This information has unique characteristics and can significantly enhance the alignment effect. The availability of entity profile information in entity alignment is analyzed, and an unsupervised entity alignment method fusing graphic and text information is proposed. This method enhances the feature representation of the entity by fusing the literal information and visual information of the entity; uses the two-way threshold nearest neighbor algorithm to filter out the entity pairs whose distance measurement is too high; uses a progressive strategy to dynamically increase the similarity threshold for controlling the quality of the generation of aligned entity pairs and their generation speed; defines the results obtained by the allocation algorithm to optimize the progressive strategy. To validate the method proposed in this paper, experiments are conducted on three sub-datasets of the DBP15K dataset, i.e. ZH_EN, JA_EN, and FR_EN. The results are compared with 10 methods including PSR, EVA, and DATTI. Experimental results show that the Hits@1 indicators reach 95.7% and 97.4% respectively on the ZH_EN and JA_EN datasets, and the Hits@10 indicator reaches 99.9% on the FR_EN dataset, showing excellent performance of the proposed method.

Key words: entity alignment, unsupervised, multimodal, progressive strategy, assignment problem

摘要： 当前的实体对齐方法，虽然利用知识图谱中实体间的结构信息取得了不错的对齐效果，但是忽略了实体间包含的大量侧面信息。这些信息具有唯一性特征，可以用于增强对齐效果。分析了实体侧面信息在实体对齐中的可用性，提出了一种无监督实体对齐方法，使用渐进式策略并融合图文信息。该方法通过融合实体的字面量信息和视觉信息，来增强实体的特征表示；采用双向阈值最近邻算法，过滤掉距离度量过高的实体对；引入渐进式策略，来动态增加相似度阈值，以控制对齐实体对的生成质量和生成速度；定义分配算法，以优化渐进式策略得到的结果。为了验证提出的方法，在DBP15K数据集的ZH_EN、JA_EN、FR_EN子数据集上进行实验，并与PSR、EVA、DATTI等10种方法的结果进行了对比分析。实验结果表明，该方法在ZH_EN和JA_EN子数据集的对齐任务上，Hits@1指标分别达到了95.7%和97.4%，在FR_EN上Hits@10指标达到了99.9%，性能表现较佳。

关键词: 实体对齐, 无监督, 多模态, 渐进式策略, 分配问题

MA He, WANG Hairong, WANG Yiyan, SUN Chong, ZHOU Beijing. Multimodal Unsupervised Entity Alignment Approach with Progressive Strategies[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(1): 245-252.

马赫, 王海荣, 王艺焱, 孙崇, 周北京. 渐进式策略的多模态无监督实体对齐方法[J]. 计算机科学与探索, 2025, 19(1): 245-252.

References

[1] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data. New York: ACM, 2008: 1247-1250.
[2] ZHU S G, CHENG X, SU S. Knowledge-based question answering by tree-to-sequence learning[J]. Neurocomputing, 2020, 372: 64-72.
[3] LOGAN R, LIU N, PETERS M, et al.Using knowledge graphs for fact-aware language modeling[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 5962-5971.
[4] YAO X C, VAN D.Information extraction over structured data: ouestion answering with freebase[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2014: 956-966.
[5] LEHMANN J, ISELE R, JAKOB M, et al. DBpedia-a large-scale, multilingual knowledge base extracted from Wikipedia[J]. Semantic Web, 2015, 6(2): 167-195.
[6] REBELE T, SUCHANEK F, HOFFART J, et al. YAGO: a multilingual knowledge base from Wikipedia, Wordnet, and Geonames[C]//Proceedings of the 16th International Semantic Web Conference, Kobe, Oct 17-21, 2016: 177-185.
[7] LIU Y, LI H, GARCIA-DURAN A, et al. MMKG: multi-modal knowledge graphs[C]//Proceedings of the 2019 European Semantic Web Conference, Portorož, Jun 2-6, 2019:459-474.
[8] 庄严, 李国良, 冯建华. 知识库实体对齐技术综述[J]. 计算机研究与发展, 2016, 53(1): 165-192.
ZHUANG Y, LI G L, FENG J H. A survey on entity alignment of knowledge base[J]. Journal of Computer Research and Development, 2016, 53(1): 165-192.
[9] 徐增林, 盛泳潘, 贺丽荣, 等. 知识图谱技术综述[J]. 电子科技大学学报, 2016, 45(4): 589-606.
XU Z L, SHENG Y P, HE L R, et al. Review on knowledge graph techniques[J]. Journal of University of Electronic Science and Technology of China, 2016, 45(4): 589-606.
[10] BORDES A, NICOLAS U, ALBERTO G D, et al. Tranlating embeddings for modeling multi-relational data[C]//Advances in Neural Information Processing Systems 26, Lake Tahoe, Dec 5-8, 2013: 2787-2795.
[11] SUN Z Q, HU W, ZHANG Q H, et al. Bootstrapping entity alignment with knowledge graph embedding[C]//Proceedings of the 2018 International Joint Conference on Artificial Intelligence, Stockholm, Jul 13-19, 2018: 4396-4402.
[12] NIGAM K, MCCALLUM A K, MITCHELL T M. Co-training for text classification[C]//Proceedings of the 9th International Conference on Information and Knowledge Management, Washington, 2000: 306-313.
[13] TRISEDYA B D,QI J Z, ZHANG R. Entity alignment between knowledge graphs using attribute embeddings[C]//Proceedings of the 33rd AAAI Conference on Artificial Intelligence, the 31st Innovative Applications of Artificial Intelligence Conference, the 9th AAAI Symposium on Educational Advances in Artificial Intelligence. Menlo Park: AAAI, 2019: 297-304.
[14] SUN Z Q, HU W, LI C K. Cross-lingual entity alignment via joint attribute-preserving embedding[C]//Proceedings of the 16th International Semantic Web Conference, 2017: 628-644.
[15] KIPF T N, WELLING M.Semi-supervised classification with graph convolutional networks[EB/OL]. [2023-08-14].https://arxiv.org/abs/1609.02907.
[16] VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al.Graph attention networks[EB/OL]. [2023-08-14]. https://arxiv.org/abs/1710.10903.
[17] WANG Z C, LV Q S, LAN X H, et al. Cross-lingual knowledge graph alignment via graph convolutional networks[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2018: 349-357.
[18] LIU F Y, CHEN M H, ROTH D, et al. Visual pivoting for (unsupervised) entity alignment[C]//Proceedings of the 35th Conference on Association for the Advancement of Artificial Intelligence. Menlo Park: AAAI, 2021: 4257-4266.
[19] MAO X, WANG W T, WU Y B, et al. Boosting the speed of entity alignment 10×: dual attention matching network with normalized hard sample mining[C]//Proceedings of the 2021 International Conference of World Wide Web. New York: ACM, 2021: 821-832.
[20] MAO X, WANG W T, WU Y B, et al. Are negative samples necessary in entity alignment? An approach with high performance, scalability and robustness[C]//Proceedings of the 30th ACM International Conference on Information & Knowledge Management. New York: ACM, 2021: 1263-1273.
[21] GAO Y J, LIU X Z, WU J Y, et al. ClusterEA: scalable entity alignment with stochastic training and normalized mini-batch similarities[C]//Proceedings of the 28th ACM SIGKDD Conference on Knowledge Discovery and Data Mining. New York: ACM, 2022: 421-431.
[22] LAMPLE G, CONNEAU A, RANNZATO M A, et al.Word translation without parallel data[C]//Proceedings of the 6th International Conference on Learning Representations, Vancouver, Apr 30-May 3, 2018.
[23] ZHAO X, ZENG W X, TANG J Y, et al. Toward entity aligment in the open world: an unsupervised approach with confidence modeling[J]. Data Science and Engineering, 2022, 7(1): 16-29.
[24] CUTURI M. Sinkhorn distances: lightspeed computation of optimal transport[C]//Advances in Neural Information Processing Systems 26, Lake Tahoe, Dec 5-8, 2013: 2292-2300.
[25] ZHANG R, TRISEDYA B D, LI M, et al. A benchmark and comprehensive survey on knowledge graph entity alignment via representation learning[J]. The VLDB Journal, 2022, 31(5): 1143-1168.
[26] 张富, 杨琳艳, 李健伟, 等. 实体对齐研究综述[J]. 计算机学报, 2022, 45(6): 1195-1225.
ZHANG F, YANG L Y, LI J W, et al. An overview of entity alignment methods[J]. Chinese Journal of Computers, 2022, 45(6): 1195-1225.
[27] 马赫, 王海荣, 周北京, 等. 基于表示学习的实体对齐方法综述[J]. 计算机工程与科学, 2023, 45(3): 554-564.
MA H, WANG H R, ZHOU B J, et al. Overview of the entity alignment methods based representation learning[J]. Computer Engineering & Science, 2023, 45(3): 554-564.
[28] MAO X, WANG W T, WU Y B, et al. From alignment to assignment: frustratingly simple unsupervised entity alignment [EB/OL]. [2023-08-14]. https://arxiv.org/abs/2109.02363.
[29] WANG C X, HUANG Z H, WAN Y, et al. FuAlign: cross-lingual entity alignment via multi-view representation learning of fused knowledge graphs[J]. Information Fusion, 2023, 89: 41-52.
[30] CHEN M H, TIAN Y T, YANG M H,et al.Multilingual knowledge graph embeddings for cross-lingual knowledge alignment[EB/OL]. [2023-08-14]. https://arxiv.org/abs/1611.03954.
[31] ZHU Y, LIU H, WU Z, et al. Relation-aware neighborhood matching model for entity alignment[C]//Proceedings of the 35th Conference on Association for the Advancement of Artificial Intelligence. Menlo Park: AAAI, 2021: 4749-4756.
[32] MAO X, MA M R, HAO Y, et al. An effective and efficient entity alignment decoding algorithm via third-order tensor isomorphism[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2022: 5888-5898.
[33] CHEN L Y, LI Z, WANG Y J, et al. MMEA: entity alignment for multi-modal knowledge graph[C]//Proceedings of the 13th International Conference on Knowledge Science, Engineering and Management, Hangzhou, Aug 28-30, 2020: 134-147.