Incorporating Subgraph Structure Knowledge Base Question Answering via Neural Reasoning

doi:10.3778/j.issn.1673-9418.2106094

Abstract

Abstract:

As an effective representation model of the real world knowledge, knowledge base (or knowledge graph) has attracted wide attention from academia and industry. In recent years, with the emergence of large-scale knowledge bases, knowledge base question answering has also attracted attention as a basic application technology of knowledge bases. Among them, the typical method based on semantic parsing transforms questions into answer retrieval on graphs by parsing query sentences, however, which neglects that there are often missing links in knowledge bases. As a result, the above process might fall short in some cases. The typical model based on neural reasoning performs entity similarity ranking by encoding questions, but it cannot solve the cold start problem of given entities in dynamic scenarios. To address the above problems, a neural inference knowledge base question-and-answer method incorporating subgraph structures is proposed to achieve a more adequate inference by taking into account the semantic and structural information of entities in the question-and-answer inference process. Firstly, the question and answer are converted into vectors containing semantic information by the pre-training model RoBERTa. Secondly, the corresponding question and answer subgraphs are constructed based on the entities in the question and answer, and the structural information of the subgraphs is extracted using graph neural networks. Then, the entity representations are pre-trained based on the background knowledge base and fused with the corresponding structural representations. Finally, the candidate answers are rated based on the fused vectors, and the entity with the highest rating is considered as the answer. Extensive experiments are conducted on the WebQuestionsSP dataset, and the experimental results show that the proposed model outperforms other benchmark models.

Key words: knowledge base question answering, neural reasoning, subgraph structure, graph convolutional network

摘要：

知识库（或知识图谱）作为一种对现实世界的有效表征模式，引起了学术界和工业界广泛关注。近年来，随着大规模知识库的出现，知识库问答技术作为知识库的基础应用技术同样备受关注。基于语义解析的代表方法通过对查询句的解析将问题转化为图上的答案检索，但知识库中往往存在缺失的链接，导致上述过程无法顺利开展；基于神经推理的代表模型通过对问题进行编码来进行实体相似度排序，但其无法解决动态场景下的实体冷启动问题。针对上述问题，提出了一种融合子图结构的神经推理式知识库问答方法，实现了在问答推理过程中兼顾实体的语义与结构信息，从而进行更充分的推理。首先，通过预训练模型RoBERTa将问句转换为包含语义的向量；其次，根据问句中的实体构建相应的问答子图，并利用图神经网络提取子图的结构信息；再次，基于背景知识库进行实体表示预训练，并与对应的结构表示进行融合；最后，根据融合后的向量对候选答案进行评分，将评分最高的实体作为答案。在WebQuestionsSP数据集上进行了对比测试，实验结果表明，提出的模型优于其他基准模型。

关键词: 知识库问答, 神经推理, 子图结构, 图卷积网络

CHEN Ziyang, LIAO Jinzhi, ZHAO Xiang, CHEN Yingguo. Incorporating Subgraph Structure Knowledge Base Question Answering via Neural Reasoning[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(10): 1870-1879.

陈子阳, 廖劲智, 赵翔, 陈盈果. 融合子图结构的神经推理式知识库问答方法[J]. 计算机科学与探索, 2021, 15(10): 1870-1879.

References

[1] MOLLá D, VICEDO J L. Question answering in restricted domains: an overview[J]. Computational Linguistics, 2007, 33(1): 41-61.
[2] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, Jun 10-12, 2008. New York: ACM, 2008: 1247-1250.
[3] BIZER C, LEHMANN J, KOBILAROV G, et al. DBpedia—a crystallization point for the Web of data[J]. Journal of Web Semantics, 2009, 7(3): 154-165.
[4] XU B, XU Y, LIANG J, et al. CN-DBpedia: a never-ending Chinese knowledge extraction system[C]//LNCS 10351: Proceedings of the 30th International Conference on Industrial Engineering and Other Applications of Applied Intelligent Systems, Arras, Jun 27-30, 2017. Cham: Springer, 2017: 428-438.
[5] CAO M Y, LI Q Q, YANG Z H, et al. A question answering system for primary liver cancer based on knowledge graph[J]. Journal of Chinese Information Processing, 2019, 33(6): 88-93.
曹明宇, 李青青, 杨志豪, 等. 基于知识图谱的原发性肝癌知识问答系统[J]. 中文信息学报, 2019, 33(6): 88-93.
[6] ZHANG Y Y, QIAN S S, FANG Q, et al. Multi-modal knowledge-aware attention network for question answering[J]. Computer Research and Development, 2020, 57(5): 1037-1045.
张莹莹, 钱胜胜, 方全, 等. 基于多模态知识感知注意力机制的问答方法[J]. 计算机研究与发展, 2020, 57(5): 1037-1045.
[7] ABUJABAL A, YAHYA M, RIEDEWALD M, et al. Automated template generation for question answering over knowledge graphs[C]//Proceedings of the 26th International Conference on World Wide Web, Perth, Apr 3-7, 2017. New York: ACM, 2017: 1191-1200.
[8] KROMPA? D, BAIER S, TRESP V. Type-constrained representation learning in knowledge graphs[C]//Proceedings of the 14th International Semantic Web Conference, Bethlehem, Oct 11-15. Cham: Springer, 2015: 640-655.
[9] DONG L, WEI F R, ZHOU M, et al. Question answering over Freebase with multi-column convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 260-269.
[10] LUO K, LIN F, LUO X, et al. Knowledge base question answering via encoding of complex query graphs[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 2185-2194.
[11] YANG B S, YIH W T, HE X D, et al. Embedding entities and relations for learning and inference in knowledge bases[J]. arXiv:1412.6575, 2014.
[12] YIH S W, CHANG M W, HE X, et al. Semantic parsing via staged query graph generation: question answering with knowledge base[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 1321-1331.
[13] YAO X, VAN DURME B. Information extraction over structured data: question answering with Freebase[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Jun 22-27, 2014. Stroudsburg: ACL, 2014: 956-966.
[14] SUN H, DHINGRA B, ZAHEER M, et al. Open domain question answering using early fusion of knowledge bases and text[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 4231-4242.
[15] SUN H, BEDRAX-WEISS T, COHEN W W. PullNet: open domain question answering with iterative retrieval on know-ledge bases and text[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 2380-2390.
[16] LI D C, ZHANG J Y, LI P. Representation learning for question classification via topic sparse autoencoder and entity embedding[C]//Proceedings of the 2018 IEEE International Conference on Big Data, Seattle, Dec 10-13, 2018. Piscataway: IEEE, 2018: 126-133.
[17] LIU Y, OTT M, GOYAL N, et al. RoBERTa: a robustly optimized BERT pretraining approach[J/OL]. (2019-07-26) [2020-06-20]. https://arxiv.org/abs/1907.11692.pdf.
[18] DIJKSTRA E W. A note on two problems in connexion with graphs[J]. Numerische Mathematik, 1959, 1(1): 269-271.
[19] SCHLICHTKRULL M, KIPF T N, BLOEM P, et al. Modeling relational data with graph convolutional networks[C]//LNCS 10843: Proceedings of the 15th European Semantic Web Conference, Heraklion, Jun 3-7, 2018. Cham: Springer, 2018: 593-607.
[20] TROUILLON T, WELBL J, RIEDEL S, et al. Complex embeddings for simple link prediction[C]//Proceedings of the 33rd International Conference on Machine Learning, New York, Jun 19-24, 2016: 2071-2080.
[21] KINGMA D P, BA J. Adam: a method for stochastic optimization[C]//Proceedings of the 3rd International Conference on Learning Representations, San Diego, May 7-9, 2015.
[22] BERANT J, CHOU A, FROSTIG R, et al. Semantic parsing on Freebase from question-answer pairs[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Oct 18-21, 2013. Stroudsburg: ACL, 2013: 1533-1544.
[23] MILLER A, FISCH A, DODGE J, et al. Key-value memory networks for directly reading documents[J]. arXiv:1606. 03126, 2016.
[24] XU K, LAI Y X, FENG Y S, et al. Enhancing key-value memory neural networks for knowledge based question answering[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 2937-2947.