集成双重语义信息和改进注意力机制的中文实体识别方法

doi:10.3778/j.issn.1673-9418.2410052

摘要/Abstract

摘要： 针对当前中文命名实体识别中数据无明显分隔符、字符连续排列导致的边界模糊和分词歧义问题，考虑从实体的双重语义信息分析和改进注意力机制入手，提出了一种集成双重语义信息和改进注意力机制的中文实体识别方法。改进多头注意力机制，通过线性门控单元有效平衡了捕获的顺序信息，并增强了多头注意力机制提供的全局信息，通过残差网络解决整体的梯度问题。在实体识别阶段的局部语义信息处理中引入知识图谱的思想，通过外部词典提取实体的局部语义特征，在全局语义信息中使用BERT捕获数据整体的语义特征，经动态融合得到双重语义信息。BiLSTM通过双向处理输入序列捕捉全面的上下文依赖信息，同时利用改进的多头注意力机制建立多个子序列，弥补了BiLSTM对于长文本句子捕捉依赖的不足。使用CRF优化标签序列的预测，得到最终的预测结果。在中文领域公开的MSRA、Weibo和人民日报数据集上进行了实验分析，实验结果表明所提方法的F1值分别为94.22%、69.96%、93.17%，较基准方法平均提高了0.95%、10.21%、1.25%，验证了所提实体识别方法在中文领域的有效性和优越性。

关键词: 实体识别, 知识图谱, 注意力机制, 双重语义

Abstract: Aiming at the problems of boundary blurring and word segmentation ambiguity caused by no obvious separator and continuous arrangement of characters in Chinese named entity recognition, a Chinese entity recognition method integrating dual semantic information and improved attention mechanism is proposed. Firstly, the multi-head attention mechanism is improved, the captured sequence information is effectively balanced by the gated linear unit, the global information provided by the multi-head attention mechanism is enhanced, and the overall gradient problem is solved by the residual network. Next, the idea of knowledge graph is introduced into the local semantic information processing in the entity recognition stage, the local semantic features of entities are extracted by external dictionary, the semantic features of the whole data are captured by BERT in the global semantic information, and the dual semantic information is obtained through dynamic fusion. Then, BiLSTM captures comprehensive context-dependent information by bidirectionally processing the input sequence, and simultaneously establishes multiple subsequences by utilizing the improved multi-head attention mechanism, which makes up for the deficiency of BiLSTM dependence on long text sentence capture. Finally, CRF is used to optimize the prediction of tag sequences to obtain the final prediction result. Experimental analysis is carried out on MSRA, Weibo and People??s Daily datasets published in Chinese domain. Experimental results show that the F1 values of the proposed method are 94.22%, 69.96% and 93.17%, respectively, which are 0.95%, 10.21% and 1.25% higher than the benchmark method on average, which verifies the effectiveness and superiority of the proposed entity recognition method in Chinese domain.

Key words: entity recognition, knowledge graph, attention mechanisms, dual semantics

冯勇, 刘明华, 王嵘冰, 徐红艳, 张永刚. 集成双重语义信息和改进注意力机制的中文实体识别方法[J]. 计算机科学与探索, 2025, 19(10): 2793-2802.

FENG Yong, LIU Minghua, WANG Rongbing, XU Hongyan, ZHANG Yonggang. Chinese Entity Recognition Method Integrating Dual Semantic Information and Improved Attention Mechanism[J]. Journal of Frontiers of Computer Science and Technology, 2025, 19(10): 2793-2802.

参考文献

[1] 王颖洁, 张程烨, 白凤波, 等. 中文命名实体识别研究综述[J]. 计算机科学与探索, 2023, 17(2): 324-341.
WANG Y J, ZHANG C Y, BAI F B, et al. Review of Chinese named entity recognition research[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(2): 324-341.
[2] LI X N, YAN H, QIU X P, et al. FLAT: Chinese NER using flat-lattice transformer[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 6836-6842.
[3] 赵山, 罗睿, 蔡志平. 中文命名实体识别综述[J]. 计算机科学与探索, 2022, 16(2): 296-304.
ZHAO S, LUO R, CAI Z P. Survey of Chinese named entity recognition[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 296-304.
[4] MIWA M, BANSAL M. End-to-end relation extraction using LSTMs on sequences and tree structures[C]//Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2016: 1105-1116.
[5] YE H B, ZHANG N Y, CHEN H, et al. Generative know-ledge graph construction: a review[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2022: 1-17.
[6] CHO K, VAN MERRIENBOER B, GULCEHRE C, et al. Learning phrase representations using RNN encoder-decoder for statistical machine translation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing. Stroudsburg: ACL, 2014: 1724-1734.
[7] CHEN Y C, GAN Z, CHENG Y, et al. Distilling knowledge learned in BERT for text generation[C]//Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. Stroudsburg: ACL, 2020: 7893-7905.
[8] GUPTA M, BENDERSKY M. Information retrieval with verbose queries[C]//Proceedings of the 38th International ACM SIGIR Conference on Research and Development in Information Retrieval. New York: ACM, 2015: 1121-1124.
[9] SUN P, YANG X Z, ZHAO X B, et al. An overview of named entity recognition[C]//Proceedings of the 2018 International Conference on Asian Language Processing. Piscataway: IEEE, 2018: 273-278.
[10] LEE L H, LU C H, LIN T M. NCUEE-NLP at SemEval-2022 task 11: Chinese named entity recognition using the BERT-BiLSTM-CRF model[C]//Proceedings of the 16th International Workshop on Semantic Evaluation. Stroudsburg: ACL, 2022: 1597-1602.
[11] MCCALLUM A, LI W. Early results for named entity recognition with conditional random fields, feature induction and web-enhanced lexicons[C]//Proceedings of the 7th Conference on Natural Language Learning. Stroudsburg: ACL, 2003: 188-191.
[12] LUO Y, XIAO F, ZHAO H. Hierarchical contextualized representation for named entity recognition[J]. Proceedings of the 34th AAAI Conference on Artificial Intelligence, 2020,34(5): 8441-8448.
[13] ZHENG C M, CAI Y, XU J Y, et al. A boundary-aware neural model for nested named entity recognition[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg: ACL, 2019: 357-366.
[14] 尹成龙, 陈爱国. 融合多重嵌入的中文命名实体识别[J]. 中文信息学报, 2023, 37(4): 63-71.
YIN C L, CHEN A G. Multi-embeddings for Chinese named entity recognition[J]. Journal of Chinese Information Processing, 2023, 37(4): 63-71.
[15] 罗辉, 卢玲. 面向中文命名实体识别的中文字符表示方法[J]. 小型微型计算机系统, 2023, 44(7): 1434-1440.
LUO H, LU L. Character embedding method for Chinese named entity recognition[J]. Journal of Chinese Computer Systems, 2023, 44(7): 1434-1440.
[16] ZHU Y, WANG G. CAN-NER: convolutional attention network for Chinese named entity recognition[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics. Stroudsburg: ACL, 2019: 3384-3393.
[17] GUI T, ZOU Y C, ZHANG Q, et al. A lexicon-based graph neural network for Chinese NER[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing. Stroudsburg: ACL, 2019: 1040-1050.
[18] WANG T B, HUANG R Y, HU N, et al. Chinese named entity recognition method based on dictionary semantic know-ledge enhancement[J]. IEICE Transactions on Information and Systems, 2023, 106(5): 1010-1017.
[19] GUO Y, FENG S X, LIU F J, et al. Enhanced Chinese domain named entity recognition: an approach with lexicon boundary and frequency weight features[J]. Applied Sciences, 2024, 14(1): 354.
[20] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies. Stroudsburg: ACL, 2019: 4171-4186.
[21] LUO R, XU J, ZHANG Y, et al. PKUSEG: a toolkit for multi-domain Chinese word segmentation[EB/OL]. [2024-08-19]. https://arxiv.org/abs/1906.11455.
[22] ZHANG S, ZHENG D, XU X, et al. Bidirectional long short-term memory networks for relation classification[C]//Proceedings of the 29th Pacific Asia Conference on Language, Information and Computation. Stroudsburg: ACL, 2015: 73-78.
[23] 叶瀚, 孙海春, 李欣. 融合GCNN与GRU的异常实体识别方法[J]. 计算机科学与探索, 2023, 17(8): 1938-1948.
YE H, SUN H C, LI X. Entity anomaly recognition method based on GCNN and GRU[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(8): 1938-1948.
[24] LI J T, MENG K. MFE-NER: multi-feature fusion embedding for Chinese named entity recognition[EB/OL]. [2024-08-19]. https://arxiv.org/abs/2109.07877.