Knowledge Graph Embedding Technology: A Review

doi:10.3778/j.issn.1673-9418.2103086

Abstract

Abstract:

Knowledge graph embedding (KGE) is a new research hotspot in the field of knowledge graphs, which aims to apply the translation invariance of word vectors to embedding entities and relationships of the knowledge graph into a low-dimensional vector space to complete knowledge representation. In this paper, it is mainly concerned with the classification according to the types of practical problems to be solved. Firstly, it expounds four major types of embedding methods of knowledge graph, including deep learning-based methods, graphical features-based methods, translation model-based methods, and other model-based methods. The algorithm ideas of each model are elaborated, and the advantages and disadvantages of each model are concluded. Secondly, the algorithm experi-ment of knowledge graph embedding is analyzed and summarized from the four aspects of commonly used data sets, evaluation indicators, algorithms, and experiments, then a horizontal and vertical comparison of the embedding method is made. Finally, from the perspective of solving practical problems, the future direction of knowledge graph embedding technology is given. Through research, it is discovered that in the deep learning-based method, LCPE achieves the best effect; in the graphical features-based method, TCE makes the best impression; whereas in the translation model-based method, NTransGH responds most optimistically. Future researches can be expanded on the basis of LCPE, TCE, and NTransGH to continuously improve the experimental effects of link prediction and triplets classification.

Key words: knowledge graph embedding (KGE), knowledge representation, knowledge graph completion (KGC), link prediction, triple classification

摘要：

知识图谱嵌入（KGE）是知识图谱领域一个新的研究热点，旨在利用词向量的平移不变性将知识图谱中实体和关系嵌入到低维向量空间，进而完成知识表示。以解决实际问题的类型为划分依据，首先，阐述了四类主要的知识图谱嵌入方法，包括基于深度学习的方法、基于图形特征的方法、基于翻译模型的方法以及基于其他模型的方法，对每种模型的算法思想进行详细阐述，总结了每种模型的优缺点；其次，从常用数据集、评价指标、算法、实验四方面对知识图谱嵌入算法实验进行分析与归纳，对嵌入方法做了横纵向对比；最后，从解决实际问题的角度出发，给出了知识图谱嵌入技术未来的发展方向。通过研究，发现在基于深度学习的方法中，LCPE模型的效果最好；在基于图形特征的方法中，TCE模型的效果最好；在基于翻译模型的方法中，NTransGH模型的效果最好。今后的研究可以在LCPE、TCE、NTransGH的基础上进行拓展，不断提高链接预测和三元组分类的实验效果。

关键词: 知识图谱嵌入（KGE）, 知识表示, 知识图谱补全（KGC）, 链接预测, 三元组分类

SHU Shitai, LI Song, HAO Xiaohong, ZHANG Liping. Knowledge Graph Embedding Technology: A Review[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(11): 2048-2062.

舒世泰，李松，郝晓红，张丽平. 知识图谱嵌入技术研究进展[J]. 计算机科学与探索, 2021, 15(11): 2048-2062.

References

[1] AMIT S. Introducing the knowledge graph[R]. America: Official Blog of Google, 2012.
[2] WIKIPEDIA. Knowledge graph[EB/OL]. [2021-03-21]. https://en.wikipedia.org/wiki/Knowledge_Graph.
[3] LI M D, SUN Z Y, ZHANG S H, et al. Enhancing knowledge graph embedding with relational constraints[J]. Neurocom- puting, 2020, 429: 33-40.
[4] LI Z F, LIU H, ZHANG Z L, et al. Recalibration convolu-tional networks for learning interaction knowledge graph embedding[J]. Neurocomputing, 2021, 427: 118-130.
[5] GONG F, WANG M, WANG H F, et al. SMR: medical know- ledge graph embedding for safe medicine recommendation[J]. Big Data Research, 2021, 23: 1-8.
[6] LU G M, ZHANG L Z, JIN M J, et al. Entity alignment via knowledge embedding and type matching constraints for knowledge graph inference[J]. Journal of Ambient Intelligence and Humanized Computing, 2021. DOI:10.1007/s12652-020-02821-2.
[7] WU Y L, ZHAO S L. Community answer generation based on knowledge graph[J]. Information Sciences, 2021, 545: 132-152.
[8] WANG J B, KUAN N, CHEN X Y, et al. SUKE: embed-ding model for prediction in uncertain knowledge graph[J]. IEEE Access, 2021, 9: 3871-3879.
[9] ZWIERZYNA M, FINAN C, DAVIES M, et al. A machine learning side effect prediction method using a comprehensive knowledge graph and network embeddings[C]//Proceedings of the British Pharmacology Society Meeting 2018, London, Dec 18-20, 2018. Hoboken: John Wiley & Sons, 2019: 3079.
[10] KIM K, HUR Y, KIM G, et al. GREG: a global level relation extraction with knowledge graph embedding[J]. Applied Sciences, 2020, 10(3): 1-12.
[11] CHAZARA P, NEGNY S, MONTASTRUC L. Flexible know-ledge representation and new similarity measure: applica-tion on case based reasoning for waste treatment[J]. Expert Systems with Applications, 2016, 58: 143-154.
[12] XIE Q Z, MA X Z, DAI Z H, et al. An interpretable know- ledge transfer model for knowledge base completion[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, ?Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 950-962.
[13] SHI B X, WENINGER T. ProjE: embedding projection for knowledge graph completion[C]//Proceedings of the 31st AAAI Conference on Artificial Intelligence, San Francisco, Feb 4-9, 2017. Menlo Park: AAAI, 2017: 1236-1242.
[14] JIANG T S, LIU T Y, GE T, et al. Encoding temporal infor-mation for time-aware link prediction[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Lan-guage Processing, Austin, Nov 1-5, 2016. Stroudsburg: ACL, 2016: 2350-2354.
[15] NGUYEN D Q, NGUYEN T D, NGUYEN D Q, et al. A novel embedding model for knowledge base completion based on convolutional neural network[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 327-333.
[16] BORDES A, USUNIER N, GARCíA-DURáN A, et al. Translating embeddings for modeling multi-relational data [C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 2787-2795.
[17] XIAO H, HUANG M L, YU H, et al. TransA: an adaptive approach for knowledge graph embedding[J]. arXiv:1509.05490, 2015.
[18] LIN Y K, LIU Z Y, SUN M S, et al. Learning entity and relation embeddings for knowledge graph completion[C]//Proceedings of the 29th AAAI Conference on Artificial Intelligence, Austin, Jan 25-30, 2015. Menlo Park: AAAI, 2015: 2181-2187.
[19] WANG Z, ZHANG J W, FENG J L, et al. Knowledge graph embedding by translating on hyperplanes[C]//Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec City, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 1112-1119.
[20] FAN M, ZHOU Q, CHANG E, et al. Transition-based know-ledge graph embedding with relational mapping properties [C]//Proceedings of the 28th Pacific Asia Conference on Language, Information and Computation, Phuket, Dec 12-14, 2014. Stroudsburg: ACL, 2014: 328-337.
[21] HE S Z, LIU K, JI G L, et al. Learning to represent know- ledge graphs with Gaussian embedding[C]//Proceedings of the 24th ACM International Conference on Information and Knowledge Management, Melbourne, Oct 19-23, 2015. New York: ACM, 2015: 623-632.
[22] BORDES A, WESTON J, COLLOBERT R, et al. Learning structured embeddings of knowledge bases[C]//Proceedings of the 25th AAAI Conference on Artificial Intelligence, San Francisco, Aug 7-11, 2011. Menlo Park: AAAI, 2011: 301-306.
[23] NICKEL M, TRESP V, KRIEGEL H. A three-way model for collective learning on multi-relational data[C]//Procee-dings of the 28th International Conference on Machine Learning, Washington, Jun 28-Jul 2, 2011. New York: ACM, 2011: 809-816.
[24] BORDES A, GLOROT X, WESTON J, et al. A semantic matching energy function for learning with multi-relational data: application to word-sense disambiguation[J]. Machine Learning, 2014, 94(2): 233-259.
[25] SOCHER R, CHEN D Q, MANNING C D, et al. Reason-ing with neural tensor networks for knowledge base com-pletion[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 926-934.
[26] XIAO H, CHEN Y D, SHI X D. Knowledge graph embed-ding based on multi-view clustering framework[J]. IEEE Transactions on Knowledge and Data Engineering, 2021, 33(2): 585-596.
[27] HUANG X, ZHANG J Y, LI D C, et al. Knowledge graph embedding based question answering[C]//Proceedings of the 12th ACM International Conference on Web Search and Data Mining, Melbourne, Feb 11-15, 2019. New York: ACM, 2019: 105-113.
[28] KANOJIA V, MAEDA H, TOGASHI R, et al. Enhancing know-ledge graph embedding with probabilistic negative sampling[C]//Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Apr 3-7, 2017. New York: ACM, 2017: 801-802.
[29] ZHANG W. Knowledge graph embedding with diversity of structures[C]//Proceedings of the 26th International Conference on World Wide Web Companion, Perth, Apr 3-7, 2017. New York: ACM, 2017: 747-753.
[30] XIA X Q, ZHANG D H, LIU Q, et al. Synergistic union of word embedding and knowledge graph for words semantic similarity measure[C]//Proceedings of the 4th International Conference on Computer and Communications, Chengdu, Dec 7-10, 2018. Piscataway: IEEE, 2018: 2349-2353.
[31] TANG X L, YUAN R, LI Q Y, et al. Timespan-aware dynamic knowledge graph embedding by incorporating temporal evo-lution[J]. IEEE Access, 2020, 8: 6849-6860.
[32] HAN X, ZHANG C H, SUN T T, et al. A triple-branch neural network for knowledge graph embedding[J]. IEEE Access, 2018, 6: 76606-76615.
[33] PEI S C, YU L, HOEHNDORF R, et al. Semi-supervised entity alignment via knowledge graph embedding with awareness of degree difference[C]//Proceedings of the 2019 World Wide Web Conference, San Francisco, May 13-17, 2019. New York: ACM, 2019: 3130-3136.
[34] ZHANG Y, CAO W H, LIU J T. A novel negative sample generating method for knowledge graph embedding[C]//Proceedings of the 2019 International Conference on Emb-edded Wireless Systems and Networks, Beijing, Feb 25-27, 2019. New York: ACM, 2019: 401-406.
[35] LEE G, KANG S, WHANG J J. Hyperlink classification via structured graph embedding[C]//Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, Paris, Jul 21-25, 2019. New York: ACM, 2019: 1017-1020.
[36] ZHANG D H, HE Y C, WANG Y P, et al. Knowledge graph embedding based on multi-information fusion[C]//Procee-dings of the 2019 IEEE International Conference on Com-puter Science and Educational Informatization, Kunming, Aug 16-19, 2019. Piscataway: IEEE, 2019: 310-314.
[37] SHAN Y C, BU C Y, LIU X J, et al. Confidence-aware negative sampling method for noisy knowledge graph embed-ding[C]//Proceedings of the 2018 IEEE International Con-ference on Big Knowledge, Singapore, Nov 17-18, 2018. Washington: IEEE Computer Society, 2018: 33-40.
[38] CHEN Q K, CHEN K, WU S, et al. Research about know-ledge graph completion based on active learning[J]. Journal of Frontiers of Computer Science and Technology, 2020, 14(5): 769-782.
陈钦况, 陈珂, 伍赛, 等. 关于主动学习下的知识图谱补全研究[J]. 计算机科学与探索, 2020, 14(5): 769-782.
[39] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning, New York, Jun 19-24, 2016: 2071-2080.
[40] DETTMERS T, MINERVINI P, STENETORP P, et al. Con-volutional 2D knowledge graph embeddings[C]//Proceedings of the 32nd AAAI Conference on Artificial Intelligence, the 30th Innovative Applications of Artificial Intelligence, and the 8th AAAI Symposium on Educational Advances in Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 1811-1818.
[41] WANG Z H, SHAO M G, LIU G J, et al. Knowledge map completion algorithm based on similar entity relationship[J]. Computer Application, 2018. DOI:10.11772/j.issn.1001-9081. 2018041238.
王子涵, 邵明光, 刘国军, 等. 基于同类实体关系的知识图谱补全算法[J]. 计算机应用, 2018. DOI:10.11772/j.issn. 1001-9081.2018041238.
[42] REN L J, LU J, GUO W. Multi-source knowledge embed-ding research of knowledge graph[C]//Proceedings of the 3rd International Conference on Circuits, Systems and Devices, Chengdu, Aug 23-25, 2019. Piscataway: IEEE, 2019: 163-166.
[43] WANG R J, WANG M, LIU J, et al. Graph embedding based query construction over knowledge graphs[C]//Proceedings of the 2018 IEEE International Conference on Big Know-ledge, Singapore, Nov 17-18, 2018. Washington: IEEE Com-puter Society, 2018: 1-8.
[44] YANG H, XIE G G, QIN Y, et al. Domain specific NMT based on knowledge graph embedding and attention[C]//Proceedings of the 21st International Conference on Advanced Communication Technology, PyeongChang, Feb 17-20, 2019. Piscataway: IEEE, 2019: 516-521.
[45] NIU G L, LI B, ZHANG Y F, et al. AutoETER: automated entity type representation with relation-aware attention for knowledge graph embedding[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2020: 1172-1181.
[46] KRIZHEVSHY A, SUTSKEVER I, HINTON G E. ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017, 60(6): 84-90.
[47] SHI J, GAO H, QI G L, et al. Knowledge graph embedding with triple context[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management, Singapore, Nov 6-10, 2017. New York: ACM, 2017: 2299-2302.
[48] ZHANG M Y, WANG Q, XU W K, et al. Discriminative path-based knowledge graph embedding for precise link prediction[C]//LNCS 10772: Proceedings of the 40th Euro-pean Conference on Information Retrieval, Grenoble, Mar 26-29, 2018. Cham: Springer, 2018: 276-288.
[49] DU W Q, LI B C, WANG R. Representation learning of knowledge graph integrating entity description and entity type[J]. Journal of Chinese Information Processing, 2020, 34(7): 50-59.
杜文倩, 李弼程, 王瑞. 融合实体描述及类型的知识图谱表示学习方法[J]. 中文信息学报, 2020, 34(7): 50-59.
[50] ZHU Q N, ZHOU X F, ZHANG P, et al. A neural translat-ing general hyperplane for knowledge graph embedding[J]. Journal of Computational Science, 2019, 30: 108-117.
[51] CHEN X J, XIANG Y. STransH: a revised translation-based model for knowledge representation[J]. Computer Science, 2019, 46(9): 184-189.
陈晓军, 向阳. STransH: 一种改进的基于翻译模型的知识表示模型[J]. 计算机科学, 2019, 46(9): 184-189.
[52] XIAO H, HUANG M L, ZHU X Y. TransG: a generative model for knowledge graph embedding[C]//Proceedings of the 54th Annual Meeting of the Association for Computa-tional Linguistics, Berlin, Aug 7-12, 2016. Stroudsburg: ACL, 2016: 2316-2325.
[53] WANG R, LI B C, HU S W, et al. Knowledge graph embed-ding via graph attenuated attention networks[J]. IEEE Access, 2020, 8: 5212-5224.
[54] GUO S, WNAG Q, WANG B, et al. SSE: semantically smooth embedding for knowledge graphs[J]. IEEE Transac-tions on Knowledge and Data Engineering, 2017, 29(4): 884-897.
[55] XIE R B, LIU Z Y, SUN M S. Representation learning of knowledge graphs with hierarchical types[C]//Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Menlo Park: AAAI, 2016: 2965-2971.
[56] XIE R B, LIU Z Y, JIA J, et al. Representation learning of knowledge graphs with entity descriptions[C]//Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Feb 12-17, 2016. Menlo Park: AAAI, 2016: 2659-2665.
[57] FENG J, HUANG M L, YANG Y, et al. GAKE: graph aware knowledge embedding[C]//Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Stroudsburg: ACL, 2016: 641-651.
[58] GUO S, WANG Q, WANG L H, et al. Jointly embedding knowledge graphs and logical rules[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 192-202.
[59] MILLER G A. WordNet: a lexical database for English[J]. Communications of the ACM, 1995, 38(11): 39-41.
[60] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, Jun 9-12, 2008. New York: ACM, 2008: 1247-1250.
[61] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Mode-ling relational data with graph convolutional networks[C]//LNCS 10843: Proceedings of the 15th International Conference on Extended Semantic Web Conference, Heraklion, Jun 3-7, 2018. Cham: Springer, 2018: 593-607.
[62] ZHANG Q J, WANG R G, JUAN Y, et al. Knowledge graph embedding by translating in time domain space for link prediction[J]. Knowledge-Based Systems, 2021, 212: 106564.
[63] LIU Z Y, SUN M S, LIN Y K, et al. Knowledge repre-sentation learning: a review[J]. Journal of Computer Research and Development, 2016, 53(2): 247-261.
刘知远, 孙茂松, 林衍凯, 等. 知识表示学习研究进展[J]. 计算机研究与发展, 2016, 53(2): 247-261.