Few-Shot Knowledge Graph Completion Based on Selective Attention

doi:10.3778/j.issn.1673-9418.2212076

Abstract

Abstract: Most few-shot knowledge graph completion models have some problems, such as low ability to learn relation representation and rarely attaching importance to the relative location and interaction between query entity pair when the relation between entities is complex or triples’ neighborhood is sparse. A selective attention mechanism and interaction awareness (SAIA) based few-shot knowledge graph completion algorithm is proposed. Firstly, by introducing selective attention mechanism in the process of aggregating neighbor information, the neighbor encoder pays more attention to important neighbors to reduce adverse effects of noise neighbors. Secondly, SAIA utilizes the information related to task relation in the background knowledge graph to learn more accurate relation embedding in the process of relationship representation learning. Finally, in order to mine the interaction information and location information between entities in knowledge graph, a common interaction rate index (CIR) of entity pair is designed to measure the degree of association between entities in 3-hop path. Then, SAIA combines entity pair semantic information to predict new fact. Experimental results show that SAIA outperforms the state-of-the-art few-shot knowledge graph completion methods. Compared with the optimal results of baseline models, the proposed method achieves 5-shot link prediction performance improvement of 0.038, 0.011, 0.028 and 0.052 on NELL-one dataset and 0.034，0.037，0.029 and 0.027 on Wiki-one dataset by the metric MRR, Hits@10, Hits@5 as well as Hits@1, which verifies the effectiveness and feasibility of SAIA.

Key words: knowledge graph, knowledge graph completion, representation learning, few-shot relation, attention mechanism

摘要： 在面对实体对关系复杂或目标邻域稀疏等情况时，现有的小样本知识图谱补全模型普遍存在关系表示学习能力不足以及忽略实体对相对位置和交互作用的问题。基于此，提出一种基于选择注意力机制和交互感知的小样本知识图谱补全模型（SAIA）。首先，通过在聚合邻域信息过程中引入选择注意机制，帮助邻域编码器聚焦更重要的邻居以减少噪声邻居的不良影响；其次，在关系表示学习阶段，利用背景知识图谱中与任务关系相关的信息学习更加准确的关系表示；最后，为了挖掘知识图谱实体之间的交互信息和位置信息，设计了一个实体对公共交互率指标（CIR）来衡量实体对三阶路径内的关联程度，然后结合实体语义信息共同预测新的事实。实验结果表明该方法优于目前最先进的小样本知识图谱补全模型。与基准模型最优的结果相比，SAIA在NELL-one和Wiki-one数据集上的5-shot链接预测中，平均倒数排名（MRR）、Hits@10、Hits@5以及Hits@1等性能评价指标分别提高了0.038、0.011、0.028和0.052以及0.034、0.037、0.029和0.027，验证了所提模型的有效性和可行性。

关键词: 知识图谱, 知识图谱补全, 表示学习, 小样本关系, 注意力机制

LIN Sui, LU Chaohai, JIANG Wenchao, LIN Xiaoshan, ZHOU Weilin. Few-Shot Knowledge Graph Completion Based on Selective Attention[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(3): 646-658.

林穗, 卢超海, 姜文超, 林晓珊, 周蔚林. 融合选择注意力的小样本知识图谱补全模型[J]. 计算机科学与探索, 2024, 18(3): 646-658.

References

[1] CHEN X, JIA S, XIANG Y. A review: knowledge reasoning over knowledge graph[J]. Expert Systems with Applications, 2020, 141(1): 112948.
[2] TAN H, ZHAO H, LI R, et al. A pipeline approach to free-description question answering in Chinese Gaokao reading comprehension[J]. Chinese Journal of Electronics, 2019, 28(1): 113-119.
[3] LEHMANN J, ISELE R, JAKOB M, et al. DBpedia—a large-scale, multilingual knowledge base extracted from Wikipedia[J]. Semantic Web, 2015, 6(2): 167-195.
[4] SAXENA A, TRIPATHI A, TALUKDAR P. Improving multi-hop question answering over knowledge graphs using know-ledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 4498-4507.
[5] WANG H, ZHANG F, ZHANG M, et al. Knowledge-aware graph neural networks with label smoothness regularization for recommender systems[C]//Proceedings of the 25th ACM SIGKDD International Conference on Knowledge Discovery Data Mining, Arkansas, Aug 4-8, 2019. New York: ACM, 2019: 968-977.
[6] NGUYEN D Q, VU T, NGUYEN T D, et al. Preparing network intrusion detection deep learning models with minimal data using adversarial domain adaptation[C]//Proceedings of the 15th ACM Asia Conference on Computer and Communications Security, Taipei, China, Oct 5-9, 2020. New York: ACM, 2020: 2180-2189.
[7] DONG X, GABRILOVICH E, HEITZ G, et al. Knowledge vault: a web-scale approach to probabilistic knowledge fusion[C]//Proceedings of the 20th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, New York, Aug 24-27, 2014. New York: ACM, 2014: 601-610.
[8] CHAMI I, WOLF A, JUAN D, et al. Low-dimensional hyperbolic knowledge graph embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 6901-6914.
[9] BORDES A, USUNIER N, GARCIA-DURAN A, et al. Translating embeddings for modeling multi-relational data[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, Nevada, Dec 5-8, 2013: 2787-2795.
[10] WANG Z, ZHANG J, FENG J, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, Quebec City, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 1112-1119.
[11] YANG B, YIH W, HE X, et al. Embedding entities and relations for learning and inference in knowledge bases[J]. arXiv: 1412.6575, 2014.
[12] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning, New York, Jun 20-22, 2016: 2071-2080.
[13] DETTMERS T, MINERVINI P, STENETORP P, et al. Convolutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, Louisiana, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 1811-1818.
[14] ZHANG N, DENG S, SUN Z, et al. Long-tail relation extraction via knowledge graph embeddings and graph convolution networks[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational, Minnesota, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 3016-3025.
[15] XIONG W, YU M, CHANG S, et al. One-shot relational learning for knowledge graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 1980-1990.
[16] ZHANG C, YAO H, HUANG C, et al. Few-shot know- ledge graph completion[C]//Proceedings of the 2020 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2020: 3041-3048.
[17] SHENG J, GUO S, CHEN Z, et al. Adaptive attentional network for few-shot knowledge graph completion[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 1681-1691.
[18] CHEN M, ZHANG W, ZHANG W, et al. Meta relational learning for few-shot link prediction in knowledge graphs[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 4216-4225.
[19] NIU G, LI Y, TANG C, et al. Relational learning with gated and attentive neighbor aggregator for few-shot knowledge graph completion[C]//Proceedings of the 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Jul 11-15, 2021. New York: ACM, 2021: 213-222.
[20] MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 26th International Conference on Neural Information Processing Systems, Nevada, Dec 5-8, 2013: 2787-2795.
[21] LIN Y, LIU Z, SUN M, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence, Texas, Jan 25-30, 2015. Menlo Park: AAAI, 2015: 2181-2187.
[22] JI G, HE S, XU L, et al. Knowledge graph embedding via dynamic mapping matrix[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 687-696.
[23] XIAO H, HUANG M, ZHU X. TransG: a generative model for knowledge graph embedding[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Aug 7-12, 2016. Stroudsburg: ACL, 2016: 2316-2325.
[24] NICKEL M, TRESP V, KRIEGEL H P. A three-way model for collective learning on multi-relational data[C]//Proceedings of the 28th International Conference on Machine Learning, New York, Jun 28-Jul 2, 2011: 809-816.
[25] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//Proceedings of the 5th Conference on Semantic Web Challenges at 15th Extended Semantic Web Conference, Crete, Jun 3-7, 2018. Cham: Springer, 2018: 593-607.
[26] VELICKOVIC P, CUCURULL G, CASANOVA A, et al. Graph attention networks[J]. arXiv:1710.10903, 2017.
[27] HE T, ZHOU H, ONG Y, et al. Not all neighbors are worth attending to: graph selective attention networks for semi-supervised learning[J]. arXiv:2210.07715, 2022.
[28] ZHANG M, WANG X, ZHU M, et al. Robust heterogeneous graph neural networks against adversarial attacks[C]//Proceedings of the 2022 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2022: 4363-4370.
[29] CHOWDHURY R R, BATHULA D R. IPNET: influential pro-totypical networks for few shot learning[J]. arXiv:2208.09345, 2022.
[30] VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, California, Dec 4-9, 2017: 5998-6008.
[31] AHMAD I, AKHTAR M U, NOOR S, et al. Missing link prediction using common neighbor and centrality based parameterized algorithm[J]. Scientific Reports, 2020, 10(1): 364-372.
[32] JAMBOR D, TERU K, PINEAU J, et al. Exploring the limits of few-shot link prediction in knowledge graphs[J]. arXiv: 2102.03419, 2021.
[33] GAO T, HAN X, LIU Z, et al. Hybrid attention-based prototypical networks for noisy few-shot relation classification[C]//Proceedings of the 2019 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2019: 6407-6414.
[34] SNELL J, SWERSKY K, ZEMEL R. Prototypical networks for few-shot learning[C]//Proceedings of the 31st International Conference on Neural Information Processing Systems, California, Dec 4-9, 2017: 4080-4090.
[35] MITCHELL T, COHEN W, HRUSCHKA E, et al. Never-ending learning[J]. Communications of the ACM, 2018, 61(5): 103-115.
[36] VRANDE?I? D, KR?TZSCH M. Wikidata: a free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[37] KAZEMI S M, POOLE D. SimplE embedding for link prediction in knowledge graphs[C]//Proceedings of the 32nd International Conference on Neural Information Processing Systems, Montreal, Dec 3-8, 2018: 4289-4300.
[38] SUN Z, DENG Z, NIE J, et al. RotatE: knowledge graph embedding by relational rotation in complex space[J]. arXiv: 1902.10197, 2019.