Survey of Open-Domain Knowledge Graph Question Answering

doi:10.3778/j.issn.1673-9418.2106095

Abstract

Abstract:

Knowledge graph question answering (KGQA) is the procedure of processing natural language questions posed by users to obtain relevant answers from knowledge graph (KG) based on some form of KG. Due to the limitation of knowledge scale, computing power and natural language processing capability, the early knowledge base question answering systems were limited to closed-domain questions. In recent years, with the development of KG and the construction of open-domain question answering (QA) datasets, KG has been used for open-domain QA research and practice. In this paper, in accordance with the development of technology, the open-domain KGQA is summarized. Firstly, five rule and template based KGQA methods are reviewed, including traditional semantic parsing, traditional information retrieval, triplet matching, utterance template, and query template. This type of methods mainly relies on manually defined rules and templates to complete QA task. Secondly, five deep learning based KGQA methods are introduced, which use neural network models to complete the subtasks of QA process, including knowledge graph embedding, memory network, neural network-based semantic parsing, neural network-based query graph, and neural network-based information retrieval method. Thirdly, four general domain KG and eleven open-domain QA datasets, which KGQA commonly used are described. Fourthly, three classic KGQA datasets are selected according to the difficulty of questions to compare and analyze the performance metric of each KGQA system, and the effect between above methods. Finally, this paper looks forward to the future research directions on this topic.

Key words: knowledge graph question answering (KGQA), open-domain, deep learning, semantic parsing, information retrieval

摘要：

知识图谱问答是通过处理用户提出的自然语言问题，基于知识图谱的某种形式，从中获取相关答案的过程。由于知识规模、计算能力及自然语言处理能力的制约，早期知识库问答系统被应用于限定领域。近年来，随着知识图谱的发展，以及开放领域问答数据集的陆续提出，知识图谱已用于开放领域问答研究与实践。以技术发展为主线，对开放领域知识图谱问答进行综述。首先，介绍五种基于规则模板的开放领域知识图谱问答方法：传统语义解析、传统信息检索、三元组匹配、话语模板和查询模板，这类方法主要依赖人工定义的规则模板完成问答工作。其次，描述五种基于深度学习的方法，这类方法采用神经网络模型完成问答过程的各类子任务，包括知识图谱嵌入、记忆网络、基于神经网络的语义解析、基于神经网络的查询图、基于神经网络的信息检索。接着，介绍开放领域知识图谱问答常用的4个通用领域知识图谱和11个开放领域问答数据集。随后，按照问题的难易程度选择3个经典问答数据集比较各问答系统的性能指标，对比不同方法间的性能差异并进行分析。最后，展望开放领域知识图谱问答的未来研究方向。

关键词: 知识图谱问答（KGQA）, 开放领域, 深度学习, 语义解析, 信息检索

CHEN Zirui, WANG Xin, WANG Lin, XU Dawei, JIA Yongzhe. Survey of Open-Domain Knowledge Graph Question Answering[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(10): 1843-1869.

陈子睿, 王鑫, 王林, 徐大为, 贾勇哲. 开放领域知识图谱问答研究综述[J]. 计算机科学与探索, 2021, 15(10): 1843-1869.

References

[1] SINGH V, DWIVEDI S K. Question answering: a survey of research, techniques and issues[J].?International Journal of Information Retrieval Research, 2014, 4(3): 14-33.
[2] GREEN B F, ALICE K W, CHOMSKY C, et al. Baseball: an automatic question-answerer[M]. New York: ACM, 1961: 219-224.
[3] WOODS W A. Progress in natural language understanding: an application to lunar geology[M]. New York: ACM, 1973: 441-450.
[4] BERNERSLEE T, HENDLER J, LASSILA O. The semantic web[J]. Scientific American, 2001, 284(5): 34-43.
[5] SHADBOLT N, HALL W, BERNERS-LEE T. The semantic web revisited[J]. IEEE Intelligent Systems, 2006, 21(3): 96-101.
[6] PUJARA J, HUI M, GETOOR L, et al. Knowledge graph identification[M]. Berlin: Springer, 2013: 542-557.
[7] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, Jun 10-12, 2008. New York: ACM, 2008: 1247-1250.
[8] AUER S, BIZER C, KOBILAROV G, et al. DBpedia: a nucleus for a web of open data[M]. Berlin: Springer, 2007: 722-735.
[9] SUCHANEK F M, KASNECI G, WEIKUM G. YAGO: a core of semantic knowledge unifying WordNet and Wikipedia[C]//Proceedings of the 16th International Conference on World Wide Web, Banff, May 2007. New York: ACM, 2007: 697-706.
[10] WOOLDRIDGE M, JENNINGS N R. Intelligent agents: theory and practice[J]. Knowledge Engineering Review, 1995, 10(2): 115-152.
[11] MLADEMIC D. Text-learning and related intelligent agents: a survey[J]. IEEE Intelligent Systems & Their Applications, 2002, 14(4): 44-54.
[12] FAST E, CHEN B, MENDELSOHN J, et al. Iris: a conver-sational agent for complex tasks[J]. arXiv:1707.05015, 2017.
[13] DIEFENBACH D, LOPEZ V, SINGH K, et al. Core techni-ques of question answering systems over knowledge bases: a survey[J]. Knowledge and Information Systems, 2017, 55(3): 529-569.
[14] CHAKRABORTY N, LUKOVNIKOV D, MAHESHWARI G, et al. Introduction to neural network based approaches for question answering over knowledge graphs[J]. arXiv:1907. 09361, 2019.
[15] FU B, QIU Y, TANG C, et al. A survey on complex question answering over knowledge base: recent advances and challenges[J]. arXiv:2007.13069, 2020.
[16] DENG C Y, ZENG G F, CAI X Q, et al. A survey of knowledge based question answering with deep learning[J]. Journal on Artificial Intelligence, 2020, 2(4): 157-166.
[17] BERANT J, CHOU A, FROSTIG R, et al. Semantic parsing on Freebase from question-answer pairs[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Oct 18-21, 2013. Stroudsburg: ACL, 2013: 1533-1544.
[18] KWIATKOWSKI T, CHOI E, ARTZI Y, et al. Scaling semantic parsers with on-the-fly ontology matching[C]//Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, Seattle, Oct 18-21, 2013. Stroudsburg: ACL, 2013: 1545-1556.
[19] BORDES A, CHOPRA S, WESTON J. Question answering with subgraph embeddings[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 615-620.
[20] YAO X C, VAN DURME B. Information extraction over structured data: question answering with Freebase[C]// Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore,?Jun 22-27, 2014. Stroudsburg: ACL, 2014: 956-966.
[21] YIH W T, HE X D, MEEK C. Semantic parsing for single-relation question answering[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Lin- guistics, Baltimore, Jun 22-27, 2014. Stroudsburg: ACL, 2014: 643-648.
[22] BAO J W, DUAN N, ZHOU M, et al. Knowledge-based question answering as machine translation[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore,?Jun 22-27, 2014. Stroudsburg: ACL, 2014: 967-976.
[23] BERANT J, LIANG P. Semantic parsing via paraphrasing[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore,?Jun 22-27, 2014.?Stroudsburg: ACL, 2014: 1415-1425.
[24] BORDES A, USUNIER N, CHOPRA S, et al. Large-scale simple question answering with memory networks[J]. arXiv:1506.02075, 2015.
[25] BAST H, HAUSSMANN E. More accurate question answering on Freebase[C]//Proceedings of the 24th ACM International on Conference on Information and Knowledge Management, Melbourne, Oct 2015. New York: ACM, 2015: 1431-1440.
[26] REDDY S, LAPATA M, STEEDMAN M. Large-scale sem-antic parsing without question-answer pairs[J]. Transactions of the Association for Computational Linguistics, 2014, 2(1): 377-392.
[27] YIH W T, CHANG M W, HE X, et al. Semantic parsing via staged query graph generation: question answering with knowledge base[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 1321-1331.
[28] XU K, REDDY S, FENG Y, et al. Question answering on Freebase via relation extraction and textual evidence[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Aug 7-12, 2016. Strou- dsburg: ACL, 2016: 2326-2336.
[29] LIANG C, BERANT J, LE Q, et al. Neural symbolic machines: learning semantic parsers on Freebase with weak supervision[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 23-33.
[30] XU K, FENG Y, HUANG S, et al. Hybrid question answering over knowledge base and free text[C]//Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Stroudsburg: ACL, 2016: 2397-2407.
[31] ANDREAS J, ROHRBACH M, DARRELL T, et al. Learning to compose neural networks for question answering[C]//Proceedings of the 2016 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, San Diego, Jun 12-17, 2016. Stroudsburg: ACL, 2016: 1545-1554.
[32] MILLER A H, FISCH A, DODGE J, et al. Key-value memory networks for directly reading documents[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016.?Stroudsburg: ACL, 2016 :?1400-1409.
[33] ZHANG Y, DAI H, KOZAREVA Z, et al. Variational reasoning for question answering with knowledge graph[J]. arXiv: 1709.04071, 2017.
[34] CUI W, XIAO Y, WANG H, et al. KBQA: learning question answering over QA corpora and knowledge bases[J]. VLDB Endowment, 2017, 10(5): 565-576.
[35] YU M, YIN W, HASAN K S, et al. Improved neural relation detection for knowledge base question answering[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 571-581.
[36] SOROKIN D, GUREVYCH I. Modeling semantics with gated graph neural networks for knowledge base question answering[C]//Proceedings of the 27th International Conference on Computational Linguistics, Santa Fe, Aug 20-26, 2018. Stroudsburg: ACL, 2018: 3306-3317.
[37] SUN H, DHINGRA B, ZAHEER M, et al. Open domain question answering using early fusion of knowledge bases and text[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 4231-4242.
[38] TALMOR A, BERANT J. The web as a knowledge-base for answering complex questions[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 641-651.
[39] XU K, LAI Y X, FENG Y S, et al. Enhancing key-value memory neural networks for knowledge based question answering[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, Jun 2019. Stroudsburg: ACL, 2019: 2937-2947.
[40] MAHESHWARI G, TRIVEDI P, LUKOVNIKOV D, et al. Learning to rank query graphs for complex question answering over knowledge graphs[C]//LNCS 11778: Proceedings of the 18th International Semantic Web Conference, Auckland, Oct 26-30, 2019. Cham: Springer, 2019: 487-504.
[41] SUN H, BEDRAX-WEISS T, COHEN W W. PullNet: open domain question answering with iterative retrieval on know-ledge bases and text[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 2380-2390.
[42] SAXENA A, TRIPATHI A, TALUKDAR P. Improving multi- hop question answering over knowledge graphs using know-ledge base embeddings[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10,?2020. Stroudsburg: ACL, 2020: 4498-4507.
[43] LAN Y S, JIANG J. Query graph generation for answering multi-hop complex questions from knowledge bases[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10,?2020. Stroudsburg: ACL, 2020: 969-974.
[44] WANG?X,?ZOU?L,?WANG?C K,?et al.?Research?on?know-ledge?graph?data?management:?a?survey[J].?Journal of Soft-ware,?2019, 30(7): 2139-2174.
王鑫, 邹磊, 王朝坤, 等. 知识图谱数据管理研究综述[J]. 软件学报, 2019, 30(7): 2139-2174.
[45] LIANG P. Lambda dependency-based compositional semantics[J]. arXiv:1309.4408, 2013.
[46] WONG Y, MOONEY R. Learning synchronous grammars for semantic parsing with lambda calculus[C]//Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics, Prague, Jun 23-30, 2007. Stroudsburg: ACL, 2007: 960-967.
[47] KWIATKOWKSI T, ZETTLEMOYER L, GOLDWATER S, et al. Inducing probabilistic CCG grammars from logical form with higher-order unification[C]//Proceedings of the 2010 Conference on Empirical Methods in Natural Language Processing, Cambridge, Oct 9-11, 2010. Stroudsburg: ACL, 2010: 1223-1233.
[48] ZETTLEMOYER L S, COLLINS M. Learning to map sentences to logical form: structured classification with probabilistic categorial grammars[J]. arXiv:1207.1420, 2012.
[49] YAHYA M, BERBERICH K, ELBASSUONI S, et al. Natural language questions for the web of data[C]//Proceedings of the 2012 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Jeju Island, Jul?12-14, 2012. Stroudsburg: ACL, 2012: 379-390.
[50] YAHYA M, BERBERICH K, SSUONI S E B, et al. Robust question answering over the web of linked data[C]//Procee-dings of the 22nd ACM International Conference on Informa-tion and Knowledge Management, New York, Oct 2013. New York: ACM, 2013: 1107-1116.
[51] BERANT J, LIANG P. Imitation learning of agenda-based semantic parsers[J]. Transactions of the Association for Computational Linguistics, 2015, 3: 545-558.
[52] REDDY S, TCKSTRM O, COLLINS M, et al. Transforming dependency structures to logical forms for semantic parsing[J]. Transactions of the Association for Computational Ling-uistics, 2016, 4(2):127-140.
[53] REDDY S, TCKSTRM O, PETROV S, et al. Universal semantic parsing[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Sep 9-11, 2017. Stroudsburg: ACL, 2017: 89-101.
[54] FADER A, ZETTLEMOYER L, ETZIONI O. Open question answering over curated and extracted knowledge bases[C]//Proceedings of the 20th ACM SIGKDD International Conf-erence on Knowledge Discovery and Data Mining, New York, Aug 24-27, 2014. New York: ACM, 2014: 1156-1165.
[55] LOPEZ V, MOTTA E. Ontology-driven question answering in AquaLog[C]//LNCS 3136: Proceedings of the 9th Inter-national Conference on Applications of Natural Languages to Information Systems, Salford, Jun 23-25, 2004. Berlin, Heidelberg: Springer, 2004: 89-102.
[56] LOPEZ V, UREN V, MOTTA E, et al. AquaLog: an ontology-driven question answering system for organizational semantic intranets[J]. Social Science Electronic Publishing, 2007, 5(2): 72-105.
[57] DAMLJANOVIC D, AGATONOVIC M, CUNNINGHAM H. Natural language interfaces to ontologies: combining synt-actic analysis and ontology-based lookup through the user interaction[M]. Berlin: Springer, 2010: 106-120.
[58] KAUFMANN E, BERNSTEIN A, FISCHER L. NLP-Reduce: a “naive” but domain-independent natural language interface for querying ontologies[C]//Proceedings of the 4th European Semantic Web Conference, Innsbruck, Jan 2007.
[59] LOPEZ V, UREN V S, SABOU M R, et al. Cross ontology query answering on the semantic web: an initial evaluation[C]//Proceedings of the 5th International Conference on Knowledge Capture, Redondo Beach, Sep 1-4, 2009. New York: ACM, 2009: 17-24.
[60] MANACHER G K. An improved version of the Cocke-Younger-Kasami algorithm[J]. Computer Languages, 1978, 3(2): 127-133.
[61] OCH F J. Minimum error rate training in statistical machine translation[C]//Proceedings of the 41st Annual Meeting of the Association for Computational Linguistics, Sapporo, Jul 7-12, 2003. Stroudsburg: ACL, 2003:?160-167.
[62] YIN P, DUAN N, KAO B, et al. Answering questions with complex semantic constraints on open knowledge bases[C]//Proceedings of the 24th ACM International Conference on Information and Knowledge Management, Melbourne, Oct 19-23, 2015. New York: ACM, 2015: 1301-1310.
[63] BOS J, CLARK S, STEEDMAN M, et al. Wide-coverage semantic representations from a CCG parser[C]//Proceedings of the 20th International Conference on Computational Lin-guistics, Geneva, Aug 23-27, 2004: 1240-1246.
[64] LEI Z, HUANG R, WANG H, et al. Natural language question answering over RDF—a graph data driven approach[C]//Proceedings of the 2014 ACM SIGMOD International Conference on Management of Data, Snowbird, Jun 22-27, 2014. New York: ACM, 2014: 313-324.
[65] SHEKARPOUR S, MARX E, NGOMO A C, et al. SINA: semantic interpretation of user queries for question answering on interlinked data[J]. Journal of Web Semantics, 2015, 30: 39-51.
[66] ZHENG W, LEI Z, XIANG L, et al. How to build templates for RDF question/answering: an uncertain graph similarity join approach[C]//Proceedings of the 2015 ACM SIGMOD International Conference on Management of Data, Melbourne, May 31-Jun 4, 2015. New York: ACM, 2015: 1809-1824.
[67] SAVENKOV D, AGICHTEIN E. When a knowledge base is not enough: question answering over knowledge bases with external text data[C]//Proceedings of the 39th International ACM SIGIR Conference on Research and Development in Information Retrieval, Pisa, Jul 17-21, 2016. New York: ACM, 2016: 235-244.
[68] CAI Q, YATES A. Large-scale semantic parsing via schema matching and lexicon extension[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Aug 4-9,?2013. Stroudsburg: ACL, 2013: 423-433.
[69] FADER A, ZETTLEMOYER L, ETZIONI O. Paraphrase-driven learning for open question answering[C]//Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics, Sofia, Aug 4-9, 2013. Stroudsburg: ACL, 2013: 1608-1618.
[70] USBECK R, NGOMO A, BUHMANN L, et al. HAWK—hybrid question answering using linked data[C]//LNCS 9088: Proceedings of the?12th European Semantic Web Conference, Portoroz, May 31-Jun 4, 2015.?Cham: Springer, 2015: 353-368.
[71] YAO X C. Lean question answering over Freebase from scratch[C]//Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics, Denver, May 31-Jun 5, 2015. Stroudsburg: ACL, 2015: 66-70.
[72] UNGER C, BUHMANN L, LEHMANN J, et al. Template-based question answering over RDF data[C]//Proceedings of the 21st International Conference on World Wide Web, Lyon, Apr 16-20, 2012. New York: ACM, 2012: 639-648.
[73] ABUJABAL A, YAHYA M, RIEDEWALD M, et al. Auto-mated template generation for question answering over knowledge graphs[C]//Proceedings of the 26th International Conference on World Wide Web, Perth, Apr 3-7,?2017. New York: ACM, 2017: 1191-1200.
[74] HU S, ZOU L, YU J X, et al. Answering natural language questions by subgraph matching over knowledge graphs[J]. IEEE Transactions on Knowledge and Data Engineering, 2017, 30(5): 824-837.
[75] BORDES A, WESTON J, USUNIER N. Open question answering with weakly supervised embedding models[C]//LNCS 8724: Proceedings of the 2014 European Conference on Machine Learning and Knowledge Discovery in Databases, Nancy, Sep 15-19, 2014.?Cham: Springer, 2014: 165-180.
[76] ZHOU M, HUANG M, ZHU X. An interpretable reasoning network for multi-relation question answering[C]//Proceedings of the 27th International Conference on Computational Lin-guistics, Santa Fe, Aug 20-26, 2018. Stroudsburg: ACL,2018: 2010-2022.
[77] SUN H, ARNOLD A O, BEDRAX-WEISS T, et al. Faithful embeddings for knowledge base queries[C]//Advances in Neural Information Processing Systems 33: Proceedings of the Annual Conference on Neural Information Processing Systems, Dec 6-12, 2020: 22505-22516.
[78] WESTON J, CHOPRA S, BORDES A. Memory networks[J]. arXiv:1410.3916, 2014.
[79] JAIN S. Question answering over knowledge base using factual memory networks[C]//Proceedings of the 2016 Con-ference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Student Research Workshop, San Diego, Jun 12-17, 2016. Stroudsburg: ACL, 2016: 109-115.
[80] HE X D, GOLUB D. Character-level question answering with attention[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 1598-1607.
[81] CHEN Z Y, LIAO J Z, ZHAO X, et al. Incorporating subgraph structure knowledge base question answering via neural reasoning[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(10): 1870-1879.
陈子阳, 廖劲智, 赵翔, 等. 融合子图结构的神经推理式知识库问答方法[J]. 计算机科学与探索, 2021, 15(10): 1870-1879.
[82] YAVUZ S, GUR I, SU Y, et al. Improving semantic parsing via answer type inference[C]//Proceedings of the 2016 Con-ference on Empirical Methods in Natural Language Proce-ssing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 149-159.
[83] DONG L, MALLINSON J, REDDY S, et al. Learning to paraphrase for question answering[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Sep 9-11, 2017. Stroudsburg: ACL, 2017: 875-886.
[84] ANSARI G A, SAHA A, KUMAR V, et al. Neural program induction for KBQA without gold programs or query annotations[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 4890-4896.
[85] SAHA A, ANSARI G A, LADDHA A, et al. Complex program induction for querying knowledge bases in the absence of gold programs[J]. Transactions of the Association for Computational Linguistics, 2019, 7: 185-200.
[86] HUA Y, LI Y F, HAFFARI G, et al. Few-shot complex knowledge base question answering via meta reinforcement learning[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20,?2020. Stroudsburg: ACL, 2020: 5827-5837.
[87] YHA D, YFL B, GQA C, et al. Less is more: data-efficient complex question answering over knowledge bases[J]. Journal of Web Semantics, 2020, 65: 100612.
[88] HU S, ZOU L, ZHANG X B. A state-transition framework to answer complex questions over knowledge base[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 2098-2108.
[89] BAO J W, NAN D, YAN Z, et al. Constraint-based question answering with knowledge graph[C]//Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Stroudsburg: ACL, 2016: 2503-2514.
[90] LUO K, LIN F, LUO X, et al. Knowledge base question answering via encoding of complex query graphs[C]// Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 2185-2194.
[91] XU K, WU L, WANG Z, et al. Exploiting rich syntactic information for semantic parsing with graph-to-sequence model[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 918-924.
[92] DONG L, WEI F R, ZHOU M, et al. Question answering over Freebase with multi-column convolutional neural networks[C]//Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, Jul 26-31, 2015. Stroudsburg: ACL, 2015: 260-269.
[93] HAO Y, ZHANG Y, KANG L, et al. An end-to-end model for question answering over knowledge base with cross-attention combining global knowledge[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, Vancouver, Jul 30-Aug 4, 2017. Stroudsburg: ACL, 2017: 221-231.
[94] MOHAMMED S, SHI P, LIN J. Strong baselines for simple question answering over knowledge graphs with and without neural networks[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Compu-tational Linguistics: Human Language Technologies, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 291-296.
[95] YANG M C, NAN D, MING Z, et al. Joint relational embed-dings for knowledge-based question answering[C]//Procee-dings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stro-udsburg: ACL, 2014:?645-650.
[96] YIN W P, YU M, XIANG B, et al. Simple question answering by attentive convolutional neural network[J]. arXiv:1606. 03391, 2016.
[97] ZHANG Y Z, LIU K, HE S Z, et al. Question answering over knowledge base with neural attention combining global knowledge information[J]. arXiv:1606.00979, 2016.
[98] GUPTA V, CHINNAKOTLA M, SHRIVASTAVA M. Retrieve and re-rank: a simple and effective IR approach to simple question answering over knowledge graphs[C]//Proceedings of the 1st Workshop on Fact Extraction and verification, 2018: 22-27.
[99] ZHANG Y Y, QIAN S S, FANG Q, et al. Multi-modal knowledge-aware attention network for question answering[J]. Journal of Computer Research and Development, 2020, 57(5): 1037-1045.
张莹莹, 钱胜胜, 方全, 等. 基于多模态知识感知注意力机制的问答方法[J]. 计算机研究与发展, 2020, 57(5): 1037-1045.
[100] LIN Q K, ZHANG L L, LIU J, et al. Question-aware graph convolutional network for educational knowledge base question answering[J]. Journal of Frontiers of Computer Science and Technology, 2021, 15(10): 1880-1887.
蔺奇卡, 张玲玲, 刘均, 等. 基于问句感知图卷积的教育知识库问答方法[J]. 计算机科学与探索, 2021, 15(10): 1880-1887.
[101] XIONG W H, YU M, CHANG S Y, et al. Improving question answering over incomplete KBs with knowledge-aware reader[J]. arXiv:1905.07098, 2019.
[102] FELLBAUM C, MILLER G. WordNet: an electronic lexical database[M]. Cambridge: MIT Press, 2000: 706-708.
[103] VRANDECIC D, KRTOETZSCH M. Wikidata: a free collaborative knowledgebase[J]. Communications of the ACM, 2014, 57(10): 78-85.
[104] SERBAN I V, GARCIA-DURAN A, GULCEHRE C, et al. Generating factoid questions with recurrent neural net-works: the 30M factoid question-answer corpus[C]//Pro-ceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Aug 7-12, 2016. Strou-dsburg: ACL, 2016: 588-598.
[105] BAO J W, DUAN N, YAN Z, et al. Constraint-based question answering with knowledge graph[C]//Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Stroudsburg: ACL, 2016: 2503-2514.
[106] YIN P, DUAN N, KAO B, et al. Answering questions with complex semantic constraints on open knowledge bases[C]//Proceedings of the 24th ACM International Conference on Information and Knowledge Management, Melbourne, Oct 19-23, 2015. New York: ACM, 2015: 1301-1310.
[107] YIH W T, RICHARDSON M, MEEK C, et al. The value of semantic parse labeling for knowledge base question answering[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics, Berlin, Aug 7-12, 2016. Stroudsburg: ACL, 2016: 201-206.
[108] TALMOR A, BERANT J. The web as a knowledge-base for answering complex questions[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics, New Orleans, Jun 1-6, 2018. Stroudsburg: ACL, 2018: 641-651.
[109] YU S, SUN H, SADLER B, et al. On generating characteristic-rich question sets for QA evaluation[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 562-572.
[110] ZHOU M, HUANG M, ZHU X. An interpretable reasoning network for multi-relation question answering[C]//Procee-dings of the 27th International Conference on Computa-tional Linguistics, Santa Fe, Aug 20-26, 2018. Strouds-burg: ACL, 2018: 2010-2022.
[111] CIMIANO P, LOPEZ V, UNGER C, et al. Multilingual question answering over linked data (QALD-3): lab over-view[C]//LNCS 8138: Proceedings of the 4th Interna-tional Conference of the Cross-Language Evaluation Forum for European Languages: Information Access Evaluation, Multilinguality, Multimodality, and Visualization, Valencia, Sep 23-26, 2013. Berlin: Springer, 2013: 321-332.
[112] UNGER C, FORASCU C, LOPEZ V, et al. Question answering over linked data (QALD-4)[C]//Working Notes for CLEF 2014 Conference, Sheffield, Sep 15-18, 2014: 1172-1180.
[113] UNGER C, FORESCU C, LOPEZ V, et al. Question answering over linked data (QALD-5)[C]//Working Notes for CLEF 2015-Conference and Labs of the Evaluation Forum, Toulouse, Sep 8-11, 2015.
[114] UNGER C, NGOMO A, CABRIO E. 6th open challenge on question answering over linked data (QALD-6)[C]// Semantic Web Challenges: 3rd SemWebEval Challenge at ESWC 2016, Heraklion, May 29-Jun 2, 2016. Cham: Springer, 2016: 171-177.
[115] USBECK R, NGOMO A, HAARMANN B, et al. Open challenge on question answering over linked data (QALD-7)[C]//Semantic Web Challenges: 4th SemWebEval Challe-nge at ESWC 2017, Portoroz, May 28-Jun 1, 2017. Cham: Springer, 2017: 59-69.
[116] TRIVEDI P, MAHESHWARI G, DUBEY M, et al. LC-QuAD: a corpus for complex question answering over knowledge graphs[C]//LNCS 10588: Proceedings of the 16th Interna-tional Semantic Web Conference, Vienna, Oct 21-25, 2017. Cham: Springer, 2017: 210-218.
[117] DUBEY M, BANERJEE D, ABDELKAWI A, et al. LC-QuAD 2.0: a large dataset for complex question answer-ing over Wikidata and DBpedia[C]//LNCS 11779: Procee-dings of the 18th International Semantic Web Conference, Auckland, Oct 26-30, 2019. Cham: Springer, 2019: 69-78.