融合多源信息的知识表示学习

doi:10.3778/j.issn.1673-9418.2009090

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (3): 591-597.DOI: 10.3778/j.issn.1673-9418.2009090

融合多源信息的知识表示学习

夏光兵, 李瑞轩⁺(), 辜希武, 刘伟

华中科技大学计算机科学与技术学院,武汉 430074

收稿日期:2020-09-29 修回日期:2021-05-10 出版日期:2022-03-01 发布日期:2021-05-26
通讯作者: + E-mail: rxli@hust.edu.cn
作者简介:夏光兵（1995—）,男,湖北黄冈人,硕士,主要研究方向为知识表示学习。
李瑞轩（1974—）,男,湖北宜昌人,博士,教授,博士生导师,主要研究方向为大数据处理与分析、云计算与边缘计算、数据挖掘与机器学习。
辜希武（1967—）,男,湖北武汉人,博士,副研究员,硕士生导师,主要研究方向为大数据处理与分析,数据挖掘与机器学习。
刘伟（1997—）,男,湖北天门人,博士研究生, 主要研究方向为自然语言处理与机器学习。
基金资助:
国家重点研发计划(2016QY01W0202);国家自然科学基金(U1836204);国家自然科学基金(U1936108)

Knowledge Representation Learning Based on Multi-source Information Combination

XIA Guangbing, LI Ruixuan⁺(), GU Xiwu, LIU Wei

School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China

Received:2020-09-29 Revised:2021-05-10 Online:2022-03-01 Published:2021-05-26
About author:XIA Guangbing, born in 1995, M.S. His research interest is knowledge representation learning.
LI Ruixuan, born in 1974, Ph.D., professor, Ph.D. supervisor. His research interests include big data processing and analysis, cloud and edge computing, data mining and machine learning.
GU Xiwu, born in 1967, Ph.D., associate pro-fessor, M.S. supervisor. His research interests include big data processing and analysis, data mining and machine learning.
LIU Wei, born in 1997, Ph.D. candidate. His re-search interests include natural language process-ing and machine learning.
Supported by:
National Key Research and Development Program of China(2016QY01W0202);National Natural Science Foundation of China(U1836204);National Natural Science Foundation of China(U1936108)

摘要/Abstract

摘要：

在知识图谱中,实体的文本描述信息、实体的层次类型信息和图的拓扑结构信息中隐藏着丰富的内容,它们可以形成对原始三元组的有效补充,帮助提高知识图谱各种任务的效果。为了充分利用这些多源异质信息,首先通过一维卷积神经网络嵌入文本描述信息,然后根据实体的层次类型信息构建投影矩阵,将三元组中的实体向量和实体的描述向量映射到特定的关系空间中来约束实体的语义信息,再基于图注意力机制融合图的拓扑结构信息,计算不同邻接点对实体的影响。在图注意力层中,计算了实体间的多跳关系来帮助改善数据稀疏的问题。最后,通过二维卷积神经网络来捕获不同维度间的全局信息,进一步提高模型的性能。链接预测实验结果表明,基于多源信息组合的知识表示学习模型（MCKRL）能够充分利用三元组以外的多源异质信息,因而相比于其他基线模型,该模型在链接预测任务上取得了更好的结果。

关键词: 知识表示学习, 实体描述, 层次类型, 拓扑结构

Abstract:

In knowledge graphs, there are rich contents hidden in the text description information of entity, the hierarchical type information of entity and the topological structure information of graph, and they can form an effective supplement to the triple information to get better performance. In order to make full use of these hetero-geneous information, the convolutional neural networks are firstly used to encode entity description. Then a projection matrix is constructed according to hierarchical type information to project entity vectors and entity description vectors into specific relation space to constrain their semantic information. After that, the graph attention mechanism is introduced to fuse the topological structure information of graph and calculate the influence of different adjacency points on entities. Meanwhile, the multi-hop relationship information between entities is calcu-lated to further solve the problem of data sparsity. Finally, a decoder is employed to capture the global information between different dimensions. Experimental results of link prediction show that the multi-source information com-bined knowledge representation learning (MCKRL) model can make good use of multi-source heterogeneous information beyond triples, so it obtains better results than other baseline models in link prediction.

Key words: knowledge representation learning, entity description, hierarchical type, topological structure

中图分类号:

TP391.1

夏光兵, 李瑞轩, 辜希武, 刘伟. 融合多源信息的知识表示学习[J]. 计算机科学与探索, 2022, 16(3): 591-597.

XIA Guangbing, LI Ruixuan, GU Xiwu, LIU Wei. Knowledge Representation Learning Based on Multi-source Information Combination[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(3): 591-597.

图/表 7

图1 MCKRL模型整体框架

Fig.1 Overall framework of MCKRL

图2 实体描述示例

Fig.2 Example of entity description

图3 CNN结构图

Fig.3 Structure of CNN

图4 层次类型示例

Fig.4 Example of hierarchy types

图5 GAT结构图

Fig.5 Structure of GAT

表1 链接预测实验结果

Table 1 Experimental results of link prediction

模型	FB15K					FB15K237
	MR	MRR	$Hits@ N$			MR	MRR	$Hits@ N$
	MR	MRR	1	3	10	MR	MRR	1	3	10
TransE^[1]	120	0.419	0.208	0.589	0.744	323	0.279	0.198	0.326	0.441
ComplEx^[8]	103	0.459	0.332	0.518	0.713	546	0.278	0.194	0.297	0.450
TKRL(WHC)^[18]	120	0.512	0.379	0.607	0.741	371	0.268	0.187	0.296	0.431
Jointly(A-LSTM)^[19]	72	0.625	0.403	0.611	0.759	292	0.281	0.203	0.355	0.462
ConvKB^[13]	68	0.697	0.524	0.764	0.811	216	0.289	0.198	0.324	0.471
Attention-Based^[15]	41	0.817	0.752	0.868	0.919	216	0.463	0.395	0.489	0.596
MCKRL	33	0.842	0.778	0.896	0.932	173	0.511	0.430	0.555	0.654

表1 链接预测实验结果

Table 1 Experimental results of link prediction

模型	FB15K					FB15K237
	MR	MRR	$Hits@ N$			MR	MRR	$Hits@ N$
	MR	MRR	1	3	10	MR	MRR	1	3	10
TransE^[1]	120	0.419	0.208	0.589	0.744	323	0.279	0.198	0.326	0.441
ComplEx^[8]	103	0.459	0.332	0.518	0.713	546	0.278	0.194	0.297	0.450
TKRL(WHC)^[18]	120	0.512	0.379	0.607	0.741	371	0.268	0.187	0.296	0.431
Jointly(A-LSTM)^[19]	72	0.625	0.403	0.611	0.759	292	0.281	0.203	0.355	0.462
ConvKB^[13]	68	0.697	0.524	0.764	0.811	216	0.289	0.198	0.324	0.471
Attention-Based^[15]	41	0.817	0.752	0.868	0.919	216	0.463	0.395	0.489	0.596
MCKRL	33	0.842	0.778	0.896	0.932	173	0.511	0.430	0.555	0.654

表2 三元组分类实验结果

Table 2 Experimental result of triple classification

模型	FB15K	FB15K237
TransE^[1]	0.955	0.903
ComplEx^[8]	0.968	0.925
TKRL(WHC)^[18]	0.965	0.912
Jointly(A-LSTM)^[19]	0.971	0.915
ConvKB^[13]	0.972	0.928
Attention-Based^[15]	0.978	0.938
MCKRL	0.980	0.943

参考文献 19

[1]	BORDES A, USUNIER N, GARCIADURAN A, et al. Tran-slating embeddings for modeling multi-relational data[C]// Proceedings of the 27th Annual Conference on Neural In-formation Processing Systems, Lake Tahoe, Dec 5-8, 2013. Cambridge: MIT Press, 2013: 2787-2795.
[2]	MIKOLOV T. Distributed representations of words and phrases and their compositionality[C]// Proceedings of the 26th International Conference on Neural Information Processing Systems, Lake Tahoe Nevada, Dec 5-10, 2013. Red Hook: Curran Associates, 2013: 3111-3119.
[3]	WANG Z, ZHANG J, FENG J, et al. Knowledge graph em-bedding by translating on hyperplanes[C]// Proceedings of the 28th AAAI Conference on Artificial Intelligence, Québec, Jul 27-31, 2014. Menlo Park: AAAI, 2014: 1112-1119.
[4]	LIN Y, LIU Z, SUN M, et al. Learning entity and relation embeddings for knowledge graph completion[C]// Procee-dings of the 32nd International Conference on Machine Lear-ning, Lille, Jul 6-11, 2015. New York: ACM, 2015: 2181-2187.
[5]	NICKEL M, TRESP V, KRIEGEL H, et al. A three-way mo-del for collective learning on multi-relational data[C]// Pro-ceedings of the 28th International Conference on Machine Learning, Bellevue, Jun 28-Jul 2, 2011. New York: ACM, 2011: 809-816.
[6]	YANG B, YIH W, HE X, et al. Embedding entities and rela-tions for learning and inference in knowledge bases[J]. arXiv:1412.6575, 2014.
[7]	NICKEL M, ROSASCO L, POGGIO T, et al. Holographic embeddings of knowledge graphs[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Feb 12-17, 2016. Menlo Park: AAAI, 2016: 1955-1961.
[8]	TROUILLON T, WELBL J, RIEDEL S, et al. Complex em-beddings for simple link prediction[C]// Proceedings of the 33rd International Conference on Machine Learning, New York, Jun 19-24, 2016. New York: ACM, 2016: 2071-2080.
[9]	LIU H, WU Y, YANG Y, et al. Analogical inference for multi-relational embeddings[C]// Proceedings of the 34th Inter-national Conference on Machine Learning, Sydney, Aug 6-11, 2017. New York: ACM, 2017: 2168-2178.
[10]	SCHLICHTKRULL M S, KIPF T, BLOEM P, et al. Mo-deling relational data with graph convolutional networks[C]// Proceedings of the 15th International Conference on Semantic Web, Heraklion, Jun 3-7, 2018. Cham: Sprin-ger, 2018: 593-607.
[11]	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[12]	SHI B, WENINGER T. Open-world knowledge graph com-pletion[C]// Proceedings of the 32nd AAAI Conference on Artificial Intelligence, New Orleans, Feb 2-7, 2018. Menlo Park: AAAI, 2018: 1957-1964.
[13]	NGUYEN D Q, NGUYEN T D, NGUYEN D Q, et al. A novel embedding model for knowledge base completion based on convolutional neural network[C]// Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 327-333.
[14]	SHANG C, TANG Y, HUANG J, et al. End-to-end structure-aware convolutional networks for knowledge base comple-tion[C]// Proceedings of the 33rd AAAI Conference on Artificial Intelligence, Honolulu, Jan 27-Feb 1, 2019. Menlo Park: AAAI, 2019: 3060-3067.
[15]	NATHANI D, CHAUHAN J, SHARMA C, et al. Learning attention-based embeddings for relation prediction in knowledge graphs[C]// Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 4710-4723.
[16]	VELIČKOVIĆ P, CUCURULL G, CASANOVA A, et al. Graph attention networks[J]. arXiv:1710.10903, 2017.
[17]	XIE R, LIU Z, JIA J, et al. Representation learning of know-ledge graphs with entity descriptions[C]// Proceedings of the 30th AAAI Conference on Artificial Intelligence, Phoenix, Feb 12-17, 2016. Menlo Park: AAAI, 2016: 2659-2665.
[18]	XIE R, LIU Z, SUN M, et al. Representation learning of knowledge graphs with hierarchical types[C]// Proceedings of the 25th International Joint Conference on Artificial Intelligence, New York, Jul 9-15, 2016. Singapore: World Scientific, 2016: 2965-2971.
[19]	XU J, QIU X, CHEN K, et al. Knowledge graph represen-tation with jointly structural and textual encoding[C]// Pro-ceedings of the 26th International Joint Conference on Ar-tificial Intelligence, Melbourne, Aug 19-25, 2017. Singapore: World Scientific, 2017: 1318-1324.

融合多源信息的知识表示学习

Knowledge Representation Learning Based on Multi-source Information Combination

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 7

参考文献 19

相关文章 9

编辑推荐

Metrics

[1]	李想, 杨兴耀, 于炯, 钱育蓉, 郑捷. 基于知识图谱卷积网络的双端推荐算法[J]. 计算机科学与探索, 2022, 16(1): 176-184.
[2]	李昌华，崔李扬，李智杰. 用于非精确图匹配的改进GCN模型[J]. 计算机科学与探索, 2020, 14(8): 1397-1408.
[3]	丁盼盼，宋国治，赵成龙，周一杰. 混合无线三维片网中无线路由节点分布研究[J]. 计算机科学与探索, 2019, 13(11): 1864-1872.
[4]	梁华林，张大坤. 三维片上网络高维超立方裂变拓扑结构研究[J]. 计算机科学与探索, 2019, 13(10): 1694-1701.
[5]	张蕾，钱峰，赵姝，陈洁，张燕平. 利用变分自编码器进行网络表示学习[J]. 计算机科学与探索, 2019, 13(10): 1733-1744.
[6]	郭晨，彭硕，王博，肖志芳，刘华. 互连网络的故障诊断研究综述[J]. 计算机科学与探索, 2018, 12(10): 1531-1546.
[7]	郑明才，赵小超，赵晋琴. 泛在互联车辆传感器网络混合拓扑结构研究[J]. 计算机科学与探索, 2016, 10(3): 363-371.
[8]	张大坤，宋国治，王莲莲，黄翠. 三维片上网络拓扑结构研究综述[J]. 计算机科学与探索, 2015, 9(2): 129-164.
[9]	莫礼平，曾水玲，周恺卿. 音形结合的方块苗文输入编码方案研究[J]. 计算机科学与探索, 2014, 8(8): 1017-1024.