SbSER：基于外部子图生成的大语言模型分步增强推理框架

doi:10.3778/j.issn.1673-9418.2409054

摘要/Abstract

摘要： 大语言模型（LLM）自问世以来在各种任务中取得了显著的成功，尤其是在机器翻译、文本生成、问答系统等任务中的卓越表现，它们的应用也迅速扩展到了更多复杂的任务中。尽管LLM在多种任务中展现了强大的能力，但在需要深入推理和逻辑推导的任务场景中，它们仍然面临显著的挑战。由于模型训练过程中依赖大量的文本数据，往往难以全面涵盖所有领域的专业知识，导致LLM在处理特定领域问题时容易产生“幻觉”问题，即输出不准确或与实际知识不符的答案。该问题可以通过在大语言模型推理中引入外部知识图谱（KG）来辅助解决。提出基于外部子图生成的大模型分步增强推理框架（SbSER）。通过生成清晰的子图Schema引导大模型完成准确的语义解析以完成问题到逻辑查询语句的转换，将知识三元组导入图数据库中以完成准确的知识查询，通过采用直接查询推理和联合推理两种推理方式实现问题的最终增强推理输出。实验表明，提出的SbSER在多个数据集上取得优异结果，显著提升了LLM在解决复杂问题上的能力。

关键词: 大语言模型, 子图生成, 分步推理

Abstract: Large language models (LLMs) have achieved significant success across various tasks, particularly in machine translation, text generation, and question-answering systems since their inception. Their applications have rapidly expanded to more complex tasks. However, despite their impressive performance in many areas, LLMs still face significant challenges in tasks that require deep reasoning and logical deduction. This is mainly due to the fact that during training, LLMs rely heavily on large volumes of textual data, which often fail to comprehensively cover specialized knowledge across all domains. As a result, LLMs tend to generate “hallucinations” in handling domain-specific problems, meaning they output inaccurate or factually incorrect answers. This issue can be mitigated by incorporating external knowledge graphs (KG) to assist in the reasoning process of LLMs. This paper presents the SbSER, a step-by-step enhanced reasoning framework for LLMs with external subgraph generation. Firstly, it guides the LLMs to do accurate semantic parsing through generating clear subgraph Schemas, converting questions into logical retrieval statements. Secondly, it imports knowledge triples into a graph database to complete precise knowledge retrieval. Finally, it achieves the final enhanced reasoning answer through the combination of two reasoning methods: direct retrieval reasoning and joint retrieval reasoning. Experiments demonstrate that the proposed SbSER achieves outstanding results across multiple datasets, significantly enhancing the ability of LLMs to solve complex problems.

Key words: large language model, subgraph generation, step-by-step reasoning

冯拓宇, 王刚亮, 乔子剑, 李伟平, 张雨松, 郭庆浪. SbSER：基于外部子图生成的大语言模型分步增强推理框架[J]. 计算机科学与探索, 2025, 19(2): 367-373.

FENG Tuoyu, WANG Gangliang, QIAO Zijian, LI Weiping, ZHANG Yusong, GUO Qinglang. SbSER: Step-by-Step Enhanced Reasoning Framework for Large Language Model with External Subgraph Generation[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(2): 367-373.

参考文献

[1] ACHIAM J, ADLER S, AGARWAL S, et al. GPT-4 technical report[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2303.08774.
[2] TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: open and efficient foundation language models[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2302.13971.
[3] HAN Z, GAO C, LIU J, et al. Parameter-efficient fine-tuning for large models: a comprehensive survey[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2403.14608.
[4] JI Z, LEE N, FRIESKE R, et al. Survey of hallucination in natural language generation[J]. ACM Computing Surveys, 2023, 55(12): 1-38.
[5] DANILEVSKY M, QIAN K, AHARONOV R, et al. A survey of the state of explainable AI for natural language processing[C]//Proceedings of the 1st Conference of the Asia-Pacific Chapter of the Association for Computational Linguistics and the 10th International Joint Conference on Natural Language Processing, Suzhou, Dec 4-7, 2020. Stroudsburg: ACL, 2020: 447-459.
[6] PAN S, LUO L, WANG Y, et al. Unifying large language models and knowledge graphs: a roadmap[J]. IEEE Transactions on Knowledge and Data Engineering, 2024, 36(7): 3580-3599.
[7] SHU Y, YU Z, LI Y, et al. TIARA: multi-grained retrieval for robust question answering over large knowledge base[C]//Proceedings of the 2022 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2022: 8108-8121.
[8] GU Y, DENG X, SU Y. Don􀆳t generate, discriminate: a proposal for grounding language models to real-world environments[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2023: 4928-4949.
[9] ZHANG Z, HAN X, LIU Z, et al. ERNIE: enhanced language representation with informative entities[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguisticss. Stroudsburg: ACL, 2019: 1441-1451.
[10] SUN Y, WANG S, FENG S, et al. ERNIE 3.0: large-scale knowledge enhanced pre-training for language understanding and generation[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2107.02137.
[11] WANG R, TANG D, DUAN N, et al. K-Adapter: infusing knowledge into pre-trained models with adapters[C]//Findings of the Association for Computational Linguistics: ACL-IJCNLP 2021. Stroudsburg: ACL, 2021: 1405-1418.
[12] WANG X, GAO T, ZHU Z, et al. KEPLER: a unified model for knowledge embedding and pre-trained language representation[J]. Transactions of the Association for Computational Linguistics, 2021, 9: 176-194.
[13] YE X, YAVUZ S, HASHIMOTO K, et al. RNG-KBQA: generation augmented iterative ranking for knowledge base question answering[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2022: 6032-6043.
[14] JIANG J, ZHOU K, DONG Z, et al. StructGPT: a general framework for large language model to reason over structured data[C]//Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2023: 9237-9251.
[15] LI X, ZHAO R, CHIA Y K, et al. Chain-of-knowledge: grounding large language models via dynamic knowledge adapting over heterogeneous sources[C]//Proceedings of the 11th International Conference on Learning Representations, Kigali, May 1-5, 2023.
[16] SUN J, XU C, TANG L, et al. Think-on-graph: deep and responsible reasoning of large language model with knowledge graph[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2307. 07697.
[17] LUO L, LI Y F, HAFFARI G, et al. Reasoning on graphs: faithful and interpretable large language model reasoning[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2310.01061.
[18] MILLER J J. Graph database applications and concepts with Neo4j[C]//Proceedings of the Southern Association for Information Systems Conference. Atlanta: AIS Electronic Library, 2013: 141-147.
[19] BAI J, BAI S, CHU Y, et al. Qwen technical report[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2309.16609.
[20] GUO A, LI X, XIAO G, et al. Spcql: a semantic parsing dataset for converting natural language into cypher[C]//Proceedings of the 31st ACM International Conference on Information & Knowledge Management. New York: ACM, 2022: 3973-3977.
[21] HU E J, SHEN Y, WALLIS P, et al. LoRA: low-rank adaptation of large language models[EB/OL]. [2024-07-25]. https://arxiv.org/abs/2106.09685.
[22] BROWN T B. Language models are few-shot learners[C]//Proceedings of the 34th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates, 2020.
[23] TALMOR A, BERANT J. The Web as a knowledge-base for answering complex questions[C]//Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: ACL, 2018: 641-651.
[24] YIH W, RICHARDSON M, MEEK C, et al. The value of semantic parse labeling for knowledge base question answering[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2016: 201-206.
[25] GU Y, KASE S, VANNI M, et al. Beyond I.I.D.: three levels of generalization for question answering on knowledege bases[C]//Proceedings of the Web Conference 2021, Ljubljana, Apr 19-23, 2021. New York: ACM, 2021: 3477-3488.
[26] BOLLACKER K, EVANS C, PARITOSH P, et al. Freebase: a collaboratively created graph database for structuring human knowledge[C]//Proceedings of the 2008 ACM SIGMOD International Conference on Management of Data, Vancouver, Jun 9-12, 2008. New York: ACM, 2008: 1247-1250.
[27] YU D, ZHANG S, NG P, et al. DecAF: joint decoding of answers and logical forms for question answering over knowledge bases[C]//Proceedings of the 11th International Conference on Learning Representations, Kigali, May 1-5, 2023.
[28] WEI J, WANG X, SCHUURMANS D, et al. Chain-of-thought prompting elicits reasoning in large language models[C]//Advances in Neural Information Processing Systems 35, New Orleans, Nov 28-Dec 9, 2022: 24824-24837.
[29] DAS R, ZAHEER M, THAI D, et al. Case-based reasoning for natural language queries over knowledge bases[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2021: 9594-9611.
[30] LI T, MA X, ZHUANG A, et al. Few-shot in-context learning on knowledge base question answering[C]//Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2023: 6966-6980.