关键词: 大语言模型, 知识推理, 相似度检索, 民众诉求, 知识图谱

Abstract: Efficiently responding to public complaints is a necessary measure to realize intelligent management and enhance public satisfaction, and the use of intelligent question answering for public complaints can save time and human resources. However, rule-based and retrieval-based models in intelligent question answering rely on preset knowledge. Therefore, they cannot provide effective responses when complaints are out of the scope of knowledge, nor can they maintain the coherence of conversations when dealing with multiple rounds of dialogues. Existing large language models can communicate smoothly with users, but general-purpose large language models lack domain knowledge. Due to the fact that the correct answers in the training data will contain information not covered by the questions, the general large language model generates wrong responses or answers that are not the questions asked, resulting in hallucination. To address these issues, a large language model (PC-LLM) for intelligent question-and-answer in the domain of public complaints has been constructed. Firstly, an entity relationship extraction model based on BERT-BiLSTM-CRF is designed to extract entities and relationships in the complaint work order in order to construct the complaint knowledge graph. The BERT model is used to vectorize the complaint work order and construct the vector index library of the complaint work order. In the stage of reply generation, this paper extracts the entities and relationships of users’ complaints, conducts knowledge reasoning through entity links in the knowledge graph of complaints, obtains potential relationship tips, and uses the knowledge graph of complaints to perform knowledge reasoning to obtain potential relationship hints. Meanwhile, this paper performs quick search of complaints within the vector index library of complaint work orders, and obtains similar complaints. Finally, a more accurate response can be generated by integrating potential relationship prompts, similar complaint prompts and complaint into a large language model. Experimental analysis shows that the performance of this large language model on the complaints dataset is significantly better than that of ChatGPT4o, ERNIE Bot, Tongyi Qianwen, and other large language models.

Key words: large language model, knowledge reasoning, similarity retrieval, public complaints, knowledge graph