计算机科学与探索

• 学术研究 •    下一篇

集成双重语义信息和改进注意力机制的中文实体识别方法

冯勇, 刘明华, 王嵘冰, 徐红艳, 张永刚   

  1. 1.辽宁大学 信息学院, 沈阳 110036
    2.辽宁大学 信息化中心, 沈阳 110036
    3.辽宁大学 网络与信息安全学院, 沈阳 110036
    4.吉林大学 符号计算与知识工程教育部重点实验室, 长春 130012

A Chinese Entity Recognition Method Integrating Dual Semantic Information and Improved Attention Mechanism

FENG Yong, LIU Minghua, WANG Rongbing, XU Hongyan, ZHANG Yonggang   

  1. 1.School of Information, Liaoning University, Shenyang 110036, China
    2.Informatization Center, Liaoning University, Shenyang 110036, China
    3.School of Cyber Science and Engineering, Liaoning University, Shenyang 110036, China
    4.Key Laboratory of Symbolic Computation and Knowledge Engineering of Ministry of Education, Jilin University, Changchun 130012, China

摘要: 针对当前中文命名实体识别中数据无明显分隔符、字符连续排列导致的边界模糊和分词歧义问题,考虑从实体的双重语义信息分析和改进注意力机制入手,提出了一种集成双重语义信息和改进注意力机制的中文实体识别方法。首先改进多头注意力机制,通过线性门控单元有效平衡了捕获的顺序信息,并增强了多头注意力机制提供的全局信息,通过残差网络解决整体的梯度问题。接下来,在实体识别阶段的局部语义信息处理中引入知识图谱的思想,通过外部词典提取实体的局部语义特征,在全局语义信息中使用BERT捕获数据整体的语义特征,经动态融合得到双重语义信息。然后,BiLSTM通过双向处理输入序列捕捉全面的上下文依赖信息,同时利用改进的多头注意力机制建立多个子序列,弥补了BiLSTM对于长文本句子捕捉依赖的不足。最后,使用CRF优化标签序列的预测,得到最终的预测结果。在中文领域公开的MSRA、Weibo和人民日报数据集上进行了实验分析,实验结果表明所提方法的F1值分别为94.22%、69.96%、93.17%,较基准方法平均提高了0.95%,9.31%,1.25%,验证了本文所提实体识别方法在中文领域的有效性和优越性。

关键词: 实体识别, 知识图谱, 注意力机制, 双重语义

Abstract: Aiming at the problems of boundary blurring and word segmentation ambiguity caused by no obvious separator and continuous arrangement of characters in Chinese named entity recognition, a Chinese entity recognition method integrating dual semantic information and improved attention mechanism is proposed by considering the analysis of dual semantic information of entities and improved attention mechanism. Firstly, the multi-head attention mechanism is improved, and the captured sequence information is effectively balanced by the Gated Linear Unit, and the global information provided by the multi-head attention mechanism is enhanced, and the overall gradient problem is solved by the Residual Network. Next, the idea of knowledge graph is introduced into the local semantic information processing in the entity recognition stage, the local semantic features of entities are extracted by external dictionary, and the semantic features of the whole data are captured by BERT in the global semantic information, and the dual semantic information is obtained through dynamic fusion. Then, BiLSTM captures comprehensive context-dependent information by bidirectionally processing the input sequence, and simultaneously establishes multiple subsequences by utilizing the improved multi-head attention mechanism, which makes up for the deficiency of BiLSTM dependence on long text sentence capture. Finally, CRF is used to optimize the prediction of tag sequences to obtain the final prediction result. Experimental analysis is carried out on MSRA, Weibo and People's Daily datasets published in Chinese domain. The experimental results show that the F1 values of the proposed method are 94.22%, 69.96% and 93.17% respectively, which are 0.95%, 9.31% and 1.25% higher than the benchmark method on average, which verifies the effectiveness and superiority of the proposed entity recognition method in Chinese domain.

Key words: Entity recognition, Knowledge graph, Attention mechanisms, Dual semantics