Named Entity Recognition Method of Large Language Model for Medical Question Answering System

doi:10.3778/j.issn.1673-9418.2307061

Abstract

Abstract: In medical question answering systems, entity recognition plays a major role. Entity recognition based on deep learning has received more and more attention. However, in the medical question answering system, due to the lack of annotated training data, deep learning methods cannot well identify discontinuous and nested entities in medical text. Therefore, a large language model-based entity recognition application method is proposed, and it is applied to the medical problem system. Firstly, the dataset related to medical question answering is processed into text that can be analyzed and processed by a large language model. Secondly, the output of the large language model is classified, and different classifications are processed accordingly. Then, the input text is used for intent recognition, and finally the results of entity recognition and intent recognition are sent to the medical knowledge graph for query, and the answer to the medical question and answer is obtained. Experiments are performed on 3 typical datasets and compared with several typical correlation methods. The results show that the method proposed in this paper performs better.

Key words: large language models, entity recognition, intent recognition, medical question and answer system

摘要： 在医疗问答系统中，实体识别发挥了重大作用。随着深度学习的发展，基于深度学习的实体识别得到了越来越多的关注。但是，在医疗问答系统中，由于缺少带标注的训练数据，深度学习方法不能够很好地识别医疗文本中的非连续实体和嵌套实体。为此，提出了一种基于大语言模型的实体识别应用方法，并且将其应用到医疗问题系统中。首先将医疗问答相关的数据集进行处理，变成大语言模型能够分析和处理的文本；其次针对大语言模型的输出进行分类，并对不同的分类采取相应的处理；然后将输入的文本进行意图识别，最终将实体识别和意图识别的结果发送到医疗知识图谱中进行查询，得到医疗问答的答案。在3个典型的数据集上进行了实验，并与几种典型的相关方法进行了对比。结果显示所提出的方法表现效果更好。

关键词: 大语言模型, 实体识别, 意图识别, 医疗问答系统

YANG Bo, SUN Xiaohu, DANG Jiayi, ZHAO Haiyan, JIN Zhi. Named Entity Recognition Method of Large Language Model for Medical Question Answering System[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(10): 2389-2402.

杨波, 孙晓虎, 党佳怡, 赵海燕, 金芝. 面向医疗问答系统的大语言模型命名实体识别方法[J]. 计算机科学与探索, 2023, 17(10): 2389-2402.

References

[1] 杨锦锋, 关毅, 何彬, 等. 中文电子病历命名实体和实体关系语料库构建[J]. 软件学报, 2016, 27(11): 2725-2746.
YANG J F, GUAN Y, HE B, et al. Corpus construction for named entities and entity relations on Chinese electronic medical records[J]. Journal of Software, 2016, 27(11): 2725-2746.
[2] WANG H, ZHANG W, ZENG Q, et al. Extracting important information from Chinese operation notes with natural language processing methods[J]. Journal of Biomedical Informatics, 2014, 48: 130-136.
[3] QUIMBAYA A P, MúNERA A S, RIVERA R A G, et al. Named entity recognition over electronic health records through a combined dictionary-based approach[J]. Procedia Computer Science, 2016, 100: 55-61.
[4] GORINSKI P, WU H, GROVER C, et al. Named entity recognition for electronic health records: a comparison of rule-based and machine learning approaches[J]. arXiv:1903. 03985v2, 2019.
[5] ZHANG S, ELHADAD N. Unsupervised biomedical named entity recognition: experiments with clinical and biological texts[J]. Journal of Biomedical Informatics, 2013, 46: 1088-1098.
[6] MCCALLUM A, LI W. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons[C]//Proceedings of the 7th Conference on Natural Language Learning at HLT-NAACL 2003. Stroudsburg: ACL, 2003: 188-191.
[7] DOS SANTOS C N, ZADROZNY B. Learning character level representations for part-of-speech tagging[C]//Proceedings of the 31st International Conference on Machine Learning, Beijing, Jun 21-26, 2014: 1818-1826.
[8] 许力, 李建华. 基于BERT和BiLSTM-CRF的生物医学命名实体识别[J]. 计算机工程与科学, 2021, 43(10): 1873-1879.
XU L, LI J H. Biomedical named entity recognition based on BERT and BiLSTM-CRF[J]. Computer Engineering & Science, 2021, 43(10): 1873-1879.
[9] SU J, MURTADHA A, PAN S, et al. Global pointer: novel efficient span-based approach for named entity recognition[J]. arXiv:2208.03054, 2022.
[10] KIM J H, WOODLAND P C. A rule-based named entity recognition system for speech input[C]//Proceedings of the 6th International Conference on Spoken Language Processing, Beijing, Oct 16-20, 2000: 528-531.
[11] HANISCH D, FUNDEL K, MEVISSEN H T, et al. ProMiner: rule-based protein and gene entity recognition[J]. BMC Bioinformatics, 2005, 6.
[12] CHEN X L, OUYANG C P, LIU Y B, et al. Improving the named entity recognition of Chinese electronic medical records by combining domain dictionary and rules[J]. International Journal of Environmental Research and Public Health, 2020, 17(8): 2687.
[13] NADEAU D, SEKINE S. A survey of named entity recognition and classification[J]. Lingvisticae Investigationes, 2007, 30: 3-26.
[14] SEKINE S, RANCHHOD E. Named entities: recognition, classification and use[M]//Benjamins Current Topics, 2009.
[15] ZHOU G, SU J. Named entity recognition using an HMM-based chunk tagger[C]//Proceedings of the 40th Annual Meeting on Association for Computational Linguistics. Stroudsburg: ACL, 2002: 473-480.
[16] SETTLES B. Biomedical named entity recognition using conditional random fields and rich feature sets[C]//Proceedings of the 2004 International Joint Workshop on Natural Language Processing in Biomedicine and Its Applications, Geneva, Aug 28-29, 2004.
[17] LIAO W, VEERAMACHANENI S. A simple semi-supervised algorithm for named entity recognition[C]//Proceedings of the NAACL HLT 2009 Workshop on Semi-Supervised Learning for Natural Language Processing. Stroudsburg: ACL, 2009.
[18] MIKHEEV A. A knowledge-free method for capitalized word disambiguation[C]//Proceedings of the 37th Annual Meeting of the Association for Computational Linguistics on Computational Linguistics. Stroudsburg: ACL, 1999.
[19] KAZAMA J, TORISAWA K. Exploiting Wikipedia as external knowledge for named entity recognition[C]//Proceedings of the 2007 Joint Conference on Empirical Methods in Natural Language Processing and Computational Natural Language Learning, Prague, Jun 28-30, 2007.
[20] TORAL A, MU?OZ R. A proposal to automatically build and maintain gazetteers for named entity recognition by using Wikipedia[C]//Proceedings of the 11th Conference of the European Chapter of the Association for Computational Linguistics. Stroudsburg: ACL, 2006.
[21] HOFFART J, YOSEF M, BORDINO I, et al. Robust disambiguation of named entities in text[C]//Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2011: 782-792.
[22] RAVIN Y, WATSON T J, WACHOLDER N. Extracting names from natural-language text: RC 20338[R]. 2000.
[23] ZHU J, UREN V, MOTTA E. ESpotter: adaptive named entity recognition for web browsing[C]//LNCS 3782: Proceedings of the 3rd Biennial Conference on Professional Knowledge Management, Kaiserslautern, Apr 10-13, 2005: 518-529.
[24] JI Z, SUN A, CONG G, et al. Joint recognition and linking of fine-grained locations from Tweets[C]//Proceedings of the 25th International Conference on World Wide Web, Montreal, Apr 11-15, 2016: 1271-1281.
[25] KRISHNAN V, MANNING C D. An effective two-stage model for exploiting non-local dependencies in named entity recognition[C]//Proceedings of the 21st International Conference on Computational Linguistics and the 44th Annual Meeting of the ACL. Stroudsburg: ACL, 2006.
[26] RABINER L, JUANG B. An introduction to hidden Markov models[J]. IEEE ASSP Magazine, 1986: 4-16.
[27] QUINLAN J R. Induction of decision trees[J]. Machine Learning, 1986: 81-106.
[28] KAPUR J N. Maximum-entropy models in science and engineering[M]. Hoboken: John Wiley & Sons, 1989.
[29] BURGES C J C. A tutorial on support vector machines for pattern recognition[J]. Data Mining and Knowledge Discovery, 1998, 2(2): 121-167.
[30] LAFFERTY J, MCCALLUM A, PEREIRA F. Conditional random fields: probabilistic models for segmenting and labeling sequence data[C]//Proceedings of the 18th International Conference on Machine Learning, Williamstown, Jun 28-Jul 1, 2001: 282-289.
[31] BIKEL D M, MILLER S, SCHWARTZ R, et al. Nymble: a high-performance learning name-finder[C]//Proceedings of the 5th Conference on Applied Natural Language Processing, Washington, Mar 31-Apr 3, 1997: 194-201.
[32] BIKEL D M, SCHWARTZ R, WEISCHEDEL R M. An algorithm that learns what??s in a name[J]. Machine Learning, 1999, 34: 211-231.
[33] MCNAMEE P, MAYFIELD J. Entity extraction without language-specific resources[C]//Proceedings of the 6th Conference on Natural Language Learning. Stroudsburg: ACL, 2002: 1-4.
[34] COLLINS M, SINGER Y. Unsupervised models for named entity classification[C]//Proceedings of the 1999 Joint SIGDAT Conference on Empirical Methods in Natural Language Processing and Very Large Corpora, College Park, Jun 21-22, 1999: 100-110.
[35] ETZIONI O, CAFARELLA M, DOWNEY D, et al. Unsupervised named-entity extraction from the Web: an experimental study[J]. Artificial Intelligence, 2005, 165(1): 91-134.
[36] NADEAU D, TURNEY P D, MATWIN S. Unsupervised named-entity recognition: generating gazetteers and resolving ambiguity[C]//Proceedings of the 19th Conference of the Canadian Society for Computational Studies of Intelligence, Québec City, Jun 7-9, 2006: 266-277.
[37] STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2017: 2670-2680.
[38] WU Y, HUANG J, XU C E, et al. Research on named entity recognition of electronic medical records based on RoBERTa and radical-level feature[J]. Wireless Communications and Mobile Computing, 2021: 2489754.
[39] WANG W, LI X, REN H, et al. Chinese clinical named entity recognition from electronic medical records based on multi-semantic features by using RoBERTa-wwm and CNN: model development and validation[J]. JMIR Medical Informatics, 2023, 11: e44597.
[40] HU Z, MA X N. A novel neural network model fusion app-roach for improving medical named entity recognition in online health expert question-answering services[J]. Expert Systems with Applications, 2023, 223: 119880.
[41] H?FER A, MOTTAHEDIN M. Minanto at SemEval-2023 Task 2: fine-tuning XLM-RoBERTa for named entity recognition on English data[C]//Proceedings of the 17th International Workshop on Semantic Evaluation, Toronto, Jul 13-14, 2023: 1127-1130.
[42] BIAN J Y, ZHENG J X, ZHANG Y Y, et al. Inspire the large language model by external knowledge on biomedical named entity recognition[J]. arXiv:2309.12278, 2023.
[43] WEI X, CUI X, CHENG N, et al. Zero-shot information extraction via chatting with ChatGPT[J]. arXiv:2302.10205, 2023.
[44] POLAK M P, MORGAN D. Extracting accurate materials data from research papers with conversational language models and prompt engineering—example of ChatGPT[J]. arXiv:2303.05352, 2023.
[45] WANG S, SUN X, LI X, et al. GPT-NER: named entity recognition via large language models[J]. arXiv:2304.10428, 2023.
[46] 蒋逸, 张伟, 王佩, 等. 基于互联网群体智能的知识图谱构造方法[J]. 软件学报, 2022, 33(7): 2646-2666.
JIANG Y, ZHANG W, WANG P, et al. Knowledge graph construction method via internet-based collective intelligence[J]. Journal of Software, 2022, 33(7): 2646-2666.
[47] ZHANG Y, YANG J. Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2018: 1554-1664.