基于大语言模型的命名实体识别研究进展

doi:10.3778/j.issn.1673-9418.2407038

摘要/Abstract

摘要： 命名实体识别旨在从非结构化的文本中识别出命名实体及类型，是问答系统、机器翻译、知识图谱构建等自然语言处理技术中一项重要的基础任务。随着人工智能技术的发展，基于大语言模型的命名实体识别技术成为一大研究热点。对基于大语言模型的命名实体识别最新研究进展进行综述，概述大语言模型和命名实体识别的发展历程，简要介绍命名实体识别任务常用的数据集和评估方法，从基于规则和字典、基于统计机器学习和基于深度学习的命名实体识别方法这三方面对目前传统命名实体识别研究工作进行梳理。按照模型架构详细阐述不同大语言模型如何应用于不同领域的命名实体识别任务，并对存在的问题和改进的方向进行分析。总结当前基于大语言模型的命名实体识别任务面临的挑战，并展望未来的研究方向。

关键词: 大语言模型, 命名实体识别, 神经网络, 深度学习

Abstract: Named entity recognition aims to identify named entities and their types from unstructured text, which is an important basic task in natural language processing technologies such as question answering system, machine translation and knowledge graph. With the development of artificial intelligence, named entity recognition based on large language model has become a hot research topic. This paper reviews the latest research progress of named entity recognition based on large language model. Firstly, the development process of large language model and named entity recognition is summarized, and the commonly used datasets and evaluation methods for named entity recognition tasks are briefly introduced. This paper sorts out the traditional research work on named entity recognition from three aspects: rule-based and dictionary-based, statistical machine learning-based and deep learning-based. Secondly, how to apply different big language models to different fields of named entity recognition tasks is described in detail according to the model architecture, and the existing problems and improvement directions are analyzed. Finally, the challenges faced by named entity recognition tasks based on big language models are summarized, and future research directions are prospected.

Key words: large language model, named entity recognition, neural network, deep learning

梁佳, 张丽萍, 闫盛, 赵宇博, 张雅雯. 基于大语言模型的命名实体识别研究进展[J]. 计算机科学与探索, 2024, 18(10): 2594-2615.

LIANG Jia, ZHANG Liping, YAN Sheng, ZHAO Yubo, ZHANG Yawen. Research Progress of Named Entity Recognition Based on Large Language Model[J]. Journal of Frontiers of Computer Science and Technology, 2024, 18(10): 2594-2615.

参考文献

[1] XU D, CHEN W, PENG W, et al. Large language models for generative information extraction: a survey[EB/OL].[2024-05-23]. https://arxiv.org/abs/2312.17617.
[2] SUN G, LIANG L, LI T, et al. Video question answering: a survey of models and datasets[J]. Mobile Networks and Applications, 2021, 26(5): 1904-1937.
[3] CHAUHAN S, DANIEL P. A comprehensive survey on various fully automatic machine translation evaluation metrics[J]. Neural Processing Letters, 2023, 55(9): 12663-12717.
[4] PENG C, XIA F, NASERIPARSA M, et al. Knowledge graphs: opportunities and challenges[J]. Artificial Intelligence Review, 2023, 56(11): 13071-13102.
[5] 罗媛媛, 杨春明, 李波, 等. 融合机器阅读理解的中文医学命名实体识别方法[J]. 计算机科学, 2023, 50(9): 287-294.
LUO Y Y, YANG C M, LI B, et al. Chinese medical named entity recognition method incorporating machine reading comprehension[J]. Computer Science, 2023, 50(9): 287-294.
[6] WANG Z, ZHANG S, ZHAO Y, et al. Risk prediction and credibility detection of network public opinion using blockchain technology[J]. Technological Forecasting and Social Change, 2023, 187: 122177.
[7] 陈光, 郭军. 大语言模型时代的人工智能: 技术内涵、行业应用与挑战[J/OL]. 北京邮电大学学报 [2024-05-23]. https:// doi.org/10.13190/j.jbupt.2024-035.
CHEN G, GUO J. Artificial intelligence in the era of large language models: technical significance, industry applications and challenges[J/OL]. Journal of Beijing University of Posts and Telecommunications [2024-05-23]. https://doi.org/10.13190/j.jbupt.2024-035.
[8] 孙凯丽, 罗旭东, 罗有容. 预训练语言模型的应用综述[J]. 计算机科学, 2023, 50(1): 176-184.
SUN K L, LUO X D, LUO Y R. Survey of applications of pretrained language models[J]. Computer Science, 2023, 50 (1): 176-184.
[9] 蔡睿, 葛军, 孙哲, 等. AI预训练大模型发展综述[J/OL]. 小型微型计算机系统 [2024-05-23]. http://kns.cnki.net/kcms/detail/21.1106.tp.20240510.1900.010.html.
CAI R, GE J, SUN Z, et al. Overview of the development of AI pre-trained large models[J/OL]. Journal of Chinese Computer Systems [2024-05-23]. http://kns.cnki.net/kcms/detail/21.1106.tp.20240510.1900.010.html.
[10] HAN K, WANG Y, CHEN H, et al. A survey on vision transformer[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2022, 45(1): 87-110.
[11] 鲁超峰, 陶冶, 文连庆, 等. 融合大语言模型和预训练模型的少量语料说话人-情感语音转换方法[J/OL]. 计算机应用 [2024-05-23]. http://kns.cnki.net/kcms/detail/51.1307.TP.20240328.2243.020.html.
LU C F, TAO Y, WEN L Q, et al. Speaker-emotion voice conversion method with limited corpus based on large language model and pre-trained model[J/OL]. Computer Applications [2024-05-23]. http://kns.cnki.net/kcms/detail/51.1307.TP. 20240328.2243.020.html.
[12] MIKOLOV T, CHEN K, CORRADO G, et al. Efficient estimation of word representations in vector space[EB/OL]. [2024-05-23]. https://arxiv.org/abs/1301.3781.
[13] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. [2024-05-23]. https://arxiv.org/abs/1810.04805.
[14] RAFFEL C, SHAZEER N, ROBERTS A, et al. Exploring the limits of transfer learning with a unified text-to-text transformer[J]. Journal of Machine Learning Research, 2020, 21: 1-67.
[15] LEWIS M, LIU Y, GOYAL N, et al. BART: denoising sequence-to-sequence pre-training for natural language generation, translation, and comprehension[EB/OL]. [2024-05-23]. https://arxiv.org/abs/1910.13461.
[16] FLORIDI L, CHIRIATTI M. GPT-3: its nature, scope, limits, and consequences[J]. Minds and Machines, 2020, 30: 681-694.
[17] SUN Y, WANG S, LI Y, et al. ERNIE 2.0: a continual pre-training framework for language understanding[C]//Proceedings of the 2020 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2020: 8968-8975.
[18] XIANG S, DONG F, XU S. A hybrid neural network based on XLNet for rumor detection[C]//Proceedings of the 2022 IEEE 2nd International Conference on Power, Electronics and Computer Applications. Piscataway: IEEE, 2022: 1207-1211.
[19] MARCUS M, SANTORINI B, MARCINKIEWICZ M A. Building a large annotated corpus of English: the Penn Treebank[J]. Computational Linguistics, 1993, 19(2): 313-330.
[20] ERXLEBEN F, GUNTHER M, KROTZSCH M, et al. Introducing wikidata to the linked data web[C]//Proceedings of the 13th International Semantic Web Conference, Riva del Garda, Oct 19-23, 2014. Cham: Springer, 2014: 50-65.
[21] RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[J]. OpenAI Blog, 2019, 1(8): 9.
[22] ZHU Y, KIROS R, ZEMEL R, et al. Aligning books and movies: towards story-like visual explanations by watching movies and reading books[C]//Proceedings of the 2015 IEEE International Conference on Computer Vision. Washington: IEEE Computer Society, 2015: 19-27.
[23] DORDEVIC D, BOZIC V, THOMMES J, et al. Rethinking attention: exploring shallow feed-forward neural networks as an alternative to attention layers in transformers (student abstract)[C]//Proceedings of the 2024 AAAI Conference on Artificial Intelligence. Menlo Park: AAAI, 2024: 23477-23479.
[24] 马昂, 于艳华, 杨胜利, 等. 基于强化学习的知识图谱综述[J]. 计算机研究与发展, 2022, 59(8): 1694-1722.
MA A, YU Y H, YANG S L, et al. Survey of knowledge graph based on reinforcement learning[J]. Journal of Computer Research and Development, 2022, 59(8): 1694-1722.
[25] 祁鹏年, 廖雨伦, 覃飙. 基于深度学习的中文命名实体识别研究综述[J]. 小型微型计算机系统, 2023, 44(9): 1857-1868.
QI P N, LIAO Y L, QIN B. Survey on deep learning for Chinese named entity recognition[J]. Journal of Chinese Computer Systems, 2023, 44(9): 1857-1868.
[26] ADAM L B, STEPHEN D P, VINCENT J D P. A maximum entropy approach to natural language processing[J]. Computational Linguistics, 1996, 22(1): 39-71．
[27] HU W, TIAN G, KANG Y, et al. Dual sticky hierarchical Dirichlet process hidden Markov model and its application to natural language description of motions[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018, 40(10): 2355-2373．
[28] CHEN P H, LIN C J, SCHOLKOPF B. A tutorial on ν-support vector machines[J]. Applied Stochastic Models in Business and Industry, 2005, 21: 111-136．
[29] LEE C, HWANG Y, OH H, et al. Fine-grained named entity recognition using conditional random fields for question answering[C]//Proceedings of the 2006 Asia Information Retrieval Symposium. Berlin, Heidelberg: Springer, 2006: 581- 587．
[30] WANG L, WANG X, LI T, et al. SAdaBoundNc: an adaptive subgradient online learning algorithm with logarithmic regret bounds[J]. Neural Computing and Applications, 2023, 35(11): 8051-8063.
[31] KRUPKA G R. SRA: description of the SRA system as used for MUC-6[C]//Proceedings of the 6th Conference on Message Understanding, Columbia, Nov 6-8, 1995. Stroud-sburg: ACL, 1995: 221-235.
[32] 王宁, 葛瑞芳, 苑春法, 等. 中文金融新闻中公司名的识别[J]. 中文信息学报, 2002, 16(2): 1-6.
WANG N, GE R F, YUAN C F, et al. Company name identification in chinese financial domain[J]. Journal of Chinese Information Processing, 2002, 16(2): 1-6.
[33] 张小衡, 王玲玲. 中文机构名称的识别与分析[J]. 中文信息学报, 1997(4): 22-33.
ZHANG X H, WANG L L. Identification and analysis of Chinese organization and institution names[J]. Journal of Chinese Information Processing, 1997(4): 22-33.
[34] BORTHWICK A, STERLING J, AGICHTEIN E, et al. NYU: description of the MENE named entity system as used in MUC-7[C]//Proceedings of the 7th Message Understanding Conference, Virginia, Apr 29-May 1, 1998. Strouds-burg: ACL, 1998: 1-7.
[35] FU G, LUKE K K. Chinese named entity recognition using lexicalized HMMs[J]. ACM SIGKDD Explorations Newsletter, 2005, 7(1): 19-25.
[36] 张华平, 刘群. 基于角色标注的中国人名自动识别研究[J]. 计算机学报, 2004, 27(1): 85-91.
ZHANG H P, LIU Q. Automatic recognition of Chinese personal name based on role tagging[J]. Chinese Journal of Computers, 2004, 27(1): 85-91.
[37] ISOZAKI H, KAZAWA H. Efficient support vector classifiers for named entity recognition[C]//Proceedings of the 19th International Conference on Computational Linguistics, Taipei, China, Aug 24-Sep 1, 2002. Stroudsburg: ACL,2002: 1-7.
[38] MCCALLUM A, LI W. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons[C]//Proceedings of the 7th Conference on Natural Language Learning, Edmonton, May 31-Jun 1, 2003. Stroudsburg: ACL, 2003: 188-191.
[39] LECUN Y, BENGIO Y, HINTON G. Deep learning[J]. Nature, 2015, 521(7553): 436-444.
[40] LI J, SUN A X, HAN J L, et al. A survey on deep learning for named entity recognition[J]. IEEE Transactions on Know-ledge and Data Engineering, 2022, 34(1): 50-70.
[41] COLLOBERT R, WESTON J, BOTTOU L, et al. Natural language processing (almost) from scratch[J]. Journal of Machine Learning Research, 2011, 12: 2493-2537.
[42] YAO L, LIU H, LIU Y, et al. Biomedical named entity recognition based on deep neutral network[J]. International Journal of Hybrid Information Technology, 2015, 8(8): 279-288.
[43] STRUBELL E, VERGA P, BELANGER D, et al. Fast and accurate entity recognition with iterated dilated convolutions[C]//Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Sep 9-11, 2017. Stroudsburg: ACL, 2017: 2670-2680.
[44] WU Y H, JIANG M, LEI J B, et al. Named entity recognition in Chinese clinical text using deep neural network[C]//Proceedings of the 15th World Congress on Health and Biomedical Informatics, S?o Paulo, Aug 19-23, 2015: 624-628.
[45] WU F Z, LIU J X, WU C H, et al. Neural Chinese named entity recognition via CNN-LSTM-CRF and joint training with word segmentation[C]//Proceedings of the 2019 World Wide Web Conference, San Francisco, May 13-17, 2019. New York: ACM, 2019: 3342-3348.
[46] GUI T, MA R T, ZHANG Q, et al. CNN-based Chinese NER with lexicon rethinking[C]//Proceedings of the 28th International Joint Conference on Artificial Intelligence, Macao, China, Aug 10-16, 2019: 4982-4988.
[47] HUANG Z H, XU W, YU K. Bidirectional LSTM-CRF models for sequence tagging[EB/OL]. [2024-05-23]. https://arxiv.org/abs/1508.01991.
[48] XU Y X, HUANG H Y, FENG C, et al. A supervised multihead self-attention network for nested named entity recognition[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, the 33rd Conference on Innovative Applications of Artificial Intelligence, the 11th Symposium on Educational Advances in Artificial Intelligence, Feb 2-9,2021. Menlo Park: AAAI, 2021: 14185-14193.
[49] ZHANG Y, YANG J. Chinese NER using lattice LSTM[C]//Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Jul 15-20, 2018. Stroudsburg: ACL, 2018: 1554-1564.
[50] LIU W, XU T G, XU Q H, et al. An encoding strategy based word-character LSTM for Chinese NER[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Jun 2-7, 2019.Stroudsburg: ACL, 2019: 2379-2389.
[51] MA R T, PENG M L, ZHANG Q, et al. Simplify the usage of lexicon in Chinese NER[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 5951-5960.
[52] CETOLI A, BRAGAGLIA S, O??HARNEY A D, et al. Graph convolutional networks for named entity recognition[C]//Proceedings of the 16th International Workshop on Treebanks and Linguistic Theories, Prague, Jan 23-24, 2018. Stroudsburg: ACL, 2018: 37-45.
[53] SUI D, CHEN Y, LIU K, et al. Leverage lexical knowlege for Chinese named entity recognition via collaborative graph network[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hongkong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 3821-3831.
[54] GUI T, ZOU Y C, ZHANG Q, et al. A lexicon-based graph neural network for Chinese NER[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hongkong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 1040-1050.
[55] TANG Z, WAN B, YANG L. Word-character graph convolution network for Chinese named entity recognition[J].IEEE/ACM Transactions on Audio, Speech, and Language Processing, 2020, 28: 1520-1532.
[56] KOROTEEV M V. BERT: a review of applications in natural language processing and understanding[EB/OL]. [2024-05-23]. https://arxiv.org/abs/2103.11943.
[57] 陈娜, 孙艳秋, 燕燕. 结合注意力机制的BERT-BiGRU-CRF中文电子病历命名实体识别[J]. 小型微型计算机系统, 2023, 44(8): 1680-1685.
CHEN N, SUN Y Q, YAN Y. Named entity recognition for Chinese electronic medical record based on BERT-BIGRU-CRF and attention mechanism[J]. Journal of Chinese Computer Systems, 2023, 44(8): 1680-1685.
[58] 陈观林, 程钊, 邹凌, 等. 基于BERT的危险化学品命名实体识别模型[J]. 广西科学, 2023, 30(1): 43-51.
CHEN G L, CHENG Z, ZOU L, et al. Identification model of named entities for hazardous chemicals based on BERT[J]. Guangxi Science, 2023, 30(1): 43-51.
[59] 马英. 《危险化学品目录(2015版)》即将实施[J]. 石油化工, 2015, 44(4): 488.
MA Y. Inventory of hazardous chemicals (2015)[J]. Petrochemical Technology, 2015, 44(4): 488.
[60] 杨盈, 邱芹军, 谢忠, 等. 人在回路学习增强的地理命名实体识别[J]. 测绘通报, 2023(8): 155-160.
YANG Y, QIU Q J, XIE Z, et al. Geographical named entity recognition based on human-in-the-loop learning enhancement[J]. Bulletin of Surveying and Mapping, 2023(8): 155-160.
[61] 李代祎, 张笑文, 严丽. 一种基于异构图网络的多模态实体识别方法[J/OL]. 小型微型计算机系统 [2024-05-23]. http://kns.cnki.net/kcms/detail/21.1106.TP.20230711.1048.002. html.
LI D Y, ZHANG X W, YAN L. A multimodal name entity recognition method based on heterogeneous graph network [J/OL]. Journal of Chinese Computer Systems [2024-05-23]. http://kns.cnki.net/kcms/detail/21.1106.TP.20230711.1048.002.html.
[62] AGRAWAL A, TRIPATHI S, VARDHAN M, et al. BERT-based transfer-learning approach for nested named-entity recognition using joint labeling[J]. Applied Sciences, 2022, 12(3): 976.
[63] LIU Y, OTT M, GOYAL N, et al. RoBERTa： a robustly optimized BERT pretraining approach[EB/OL]. [2024-05-23].https://arxiv.org/abs/1907.11692.
[64] LAN Z, CHEN M, GOODMAN S, et al. ALBERT: a lite BERT for self-supervised learning of language representations[C]//Proceedings of the 2020 International Conference on Learning Representations, Addis Ababa, Apr 26-30, 2020.
[65] JIAO X, YIN Y, SHANG L, et al. TinyBERT: distilling BERT for natural language understanding[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 4163-4174.
[66] JOSHI M, CHEN D, LIU Y, et al. SpanBERT: improving pre-training by representing and predicting spans[J]. Transactions of the Association for Computational Linguistics, 2020, 8: 64-77.
[67] ZHANG Z, HAN X, LIU Z, et al. ERNIE: enhanced language representation with informative entities[EB/OL]. [2024- 05-23]. https://arxiv.org/abs/1905.07129.
[68] 贾李睿智, 刘胜全, 刘源, 等. 基于分层ERNIE模型的中文嵌套命名实体识别[J]. 东北师大学报(自然科学版), 2023, 55(1): 97-103.
JIALI R Z, LIU S Q, LIU Y, et al. Chinese nested named entity recognition based on layered ERNIE model[J]. Journal of North east Normal University (Natural Science Edition), 2023, 55(1): 97-103.
[69] 王刘坤, 李功权. 基于GeoERNIE-BiLSTM-Attention-CRF模型的地质命名实体识别[J]. 地质科学, 2023, 58(3): 1164-1177.
WANG L K, LI G Q. Geological named entity recognition based on GeoERNIE-BiLSTM-Attention-CRF model[J]. Chinese Journal of Geology, 2023, 58(3): 1164-1177.
[70] 罗峦, 夏骄雄. 融合ERNIE与改进Transformer的中文NER模型[J]. 计算机技术与发展, 2022, 32(10): 120-125.
LUO L, XIA J X. Research on Chinese named entity recognition combining ERNIE with improved Transformer[J]. Computer Technology and Development, 2022, 32(10): 120-125.
[71] WANG Y, SUN Y, MA Z, et al. An ERNIE-based joint model for Chinese named entity recognition[J]. Applied Sciences, 2020, 10(16): 5711.
[72] LIU X, ZHENG Y, DU Z, et al. GPT understands, too[J/OL]. AI Open [2024-05-23]. https://doi.org/10.1016/j.aiopen.2023. 08.012.
[73] RADFORD A. Improving language understanding by generative pre-training[EB/OL]. [2024-05-23]. https://cdn.openai.com/research-covers/language-unsupervised/language_ understanding_paper.pdf.
[74] QU Y, LIU P, SONG W, et al. A text generation and prediction system: pre-training on new corpora using BERT and GPT-2[C]//Proceedings of the 2020 IEEE 10th International Conference on Electronics Information and Emergency Communication. Piscataway: IEEE, 2020: 323-326.
[75] BROWN T, MANN B, RYDER N, et al. Language models are few-shot learners[C]//Advances in Neural Information Processing Systems 33, Dec 6-12, 2020: 1877-1901.
[76] LI J, ZHOU Y, JIANG X, et al. Are synthetic clinical notes useful for real natural language processing tasks: a case study on clinical entity recognition[J]. Journal of the American Medical Informatics Association, 2021, 28(10): 2193-2201.
[77] 鲍彤, 章成志. ChatGPT中文信息抽取能力测评——以三种典型的抽取任务为例[J]. 数据分析与知识发现, 2023, 7(9): 1-11.
BAO T, ZHANG C Z. Extracting Chinese information with ChatGPT: an empirical study by three typical tasks[J]. Data Analysis and Knowledge Discovery, 2023, 7(9): 1-11.
[78] HU Y, MAI G, CUNDY C, et al. Geo-knowledge-guided GPT models improve the extraction of location descriptions from disaster-related social media messages[J]. International Journal of Geographical Information Science, 2023, 37(11): 2289-2318.
[79] HU Y, CHEN Q, DU J, et al. Improving large language models for clinical named entity recognition via prompt engineering[J/OL]. Journal of the American Medical Informatics Association [2024-05-23]. https://doi.org/10.1093/jamia/ocad259.
[80] TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: open and efficient foundation language models[EB/OL].[2024-05-23]. https://arxiv.org/abs/2302.13971.
[81] TOUVRON H, MARTIN L, STONE K, et al. LLaMA 2: open foundation and fine-tuned chat models[EB/OL]. [2024- 05-23]. https://arxiv.org/abs/2307.09288.
[82] YOON J, GUPTA A, ANUMANCHIPALLI G. Is bigger edit batch size always better?—an empirical study on model editing with LLaMA-3[EB/OL]. [2024-05-23]. https://arxiv.org/abs/2405.00664.
[83] KELOTH V K, HU Y, XIE Q, et al. Advancing entity recognition in biomedicine via instruction tuning of large language models[J]. Bioinformatics, 2024, 40(4).
[84] DE-FITERO-DOMINGUEZ D, GARCIA-LOPEZ E, GARCIA-CABOT A, et al. Distractor generation through text-to-text transformer models[J]. IEEE Access, 2024, 12: 25580-25589.
[85] PIRES T, SCHLINGER E, GARRETTE D. How multilingual is multilingual BERT[EB/OL]. [2024-05-23]. https://arxiv.org/abs/1906.01502.
[86] SCHWENK H, LI X. A corpus for multilingual document classification in eight languages[EB/OL]. [2024-05-23]. https://arxiv.org/abs/1805.09821.
[87] CHOURE A A, ADHAO R B, PACHGHARE V K. NER in Hindi language using transformer model: XLM-RoBERTa[C]//Proceedings of the 2022 IEEE International Conference on Blockchain and Distributed Systems Security. Piscataway: IEEE, 2022: 1-5.
[88] LIU C L, HSU T Y, CHUANG Y S, et al. A study of cross-lingual ability and language-specific information in multilingual BERT[EB/OL]. [2024-05-23]. https://arxiv.org/abs/2004.09205.
[89] 徐月梅, 曹晗, 王文清, 等. 跨语言情感分析研究综述[J]. 数据分析与知识发现, 2023, 7(1): 1-21.
XU Y M, CAO H, WANG W Q, et al. Cross-lingual sentiment analysis: a survey[J]. Data Analysis and Knowledge Discovery, 2023, 7(1): 1-21.
[90] MOEZZI S A R, GHAEDI A, RAHMANIAN M, et al. Application of deep learning in generating structured radiology reports: a transformer-based technique[J]. Journal of Digital Imaging, 2023, 36(1): 80-90.
[91] JI L, YAN D, CHENG Z, et al. Improving unified named entity recognition by incorporating mention relevance[J]. Neural Computing and Applications, 2023, 35(30): 22223-22234.
[92] CUI L, WU Y, LIU J, et al. Template-based named entity recognition using BART[EB/OL]. [2024-05-23]. https://arxiv.org/abs/2106.01760.
[93] DU H, XU J, DU Z, et al. MF-MNER: multi-models fusion for MNER in Chinese clinical electronic medical records[J]. Interdisciplinary Sciences: Computational Life Sciences, 2024, 16: 489-502.
[94] ANG Z, DAI Z, YANG Y, et al. XLNET: generalized autoregressive pretraining for language understanding[C]//Advances in Neural Information Processing Systems 32, Vancouver, Dec 8-14, 2019: 5754-5764.
[95] 姚贵斌, 张起贵. 基于XLnet语言模型的中文命名实体识别[J]. 计算机工程与应用, 2021, 57(18): 156-162.
YAO G B, ZHANG Q G. Chinese named entity recognition based on XLnet language model[J]. Computer Engineering and Applications, 2021, 57(18): 156-162.
[96] 沈宙锋, 苏前敏, 郭晶磊. 基于XLNet-BiLSTM的中文电子病历命名实体识别方法[J]. 智能计算机与应用, 2021, 11(8): 97-102.
SHEN Z F, SU Q M, GUO J L. Named entity recognition model of Chinese clinical electronic medical record based on XLNet-BiLSTM[J]. Intelligent Computer and Applications, 2021, 11(8): 97-102.
[97] 陈明, 顾凡. 基于XLNet的农业命名实体识别方法[J]. 河北农业大学学报, 2023(4): 111-117.
CHEN M, GU F. Agricultural named entity recognition method based on XLNet[J]. Journal of Hebei Agricultural University, 2023(4): 111-117.
[98] YAN R, JIANG X, DANG D. Named entity recognition by using XLNet-BiLSTM-CRF[J]. Neural Processing Letters, 2021, 53(5): 3339-3356.
[99] CHAI Z, JIN H, SHI S, et al. Noise reduction learning based on XLNet-CRF for biomedical named entity recognition[J]. IEEE/ACM Transactions on Computational Biology and Bioinformatics, 2022, 20(1): 595-605.
[100] CHAI Z, JIN H, SHI S, et al. Hierarchical shared transfer learning for biomedical named entity recognition[J]. BMC Bioinformatics, 2022, 23: 1-14.