计算机科学与探索

• 学术研究 •    

双特征融合注意力的方面情感分析模型

李锦,夏鸿斌,刘渊   

  1. 1.江南大学 人工智能与计算机学院,江苏 无锡 214122
    2.江苏省媒体设计与软件技术重点实验室,江苏 无锡 214122

Dual Features Local-Global Attention Model with BERT

LI Jin, XIA Hongbin, LIU Yuan   

  1. 1. School of Artificial Intelligence and Computer, Jiangnan University, Wuxi, Jiangsu 214122, China
    2. Jiangsu Key Laboratory of Media Design and Software Technology, Wuxi, Jiangsu 214122, China

摘要: 方面情感分析旨在预测句子或文档中一个特定方面的情感极性,现阶段大部分的研究都是使用注意力机制对上下文进行建模。然而,目前情感分类模型在使用BERT模型计算表征之间的依赖关系抽取特征时,大多未根据不同的语境背景考虑上下文信息,而导致建模后的特征缺乏上下文的语境信息。同时,方面词的重要性未能得到充分的重视而影响模型整体分类的性能。针对上述问题,本文提出基于双特征融合注意力的方面情感分析模型(Dual Features Local-Global Attention with BERT,DFLGA-BERT),分别设计了局部与全局的特征抽取模块,充分捕捉方面词和上下文的语义关联。并将一种改进的“准”注意力添加到DFLGA-BERT的全局特征抽取器中,使模型学习在注意力的融合中使用减性注意力以削弱本文噪声产生的负面影响。基于条件层规泛化(Conditional Layer Normalization,CLN)设计了局部特征和全局特征的特征融合结构来更好地融合局部和全局特征。在SentiHood和SemEval 2014 Task 4数据集上进行了实验。实验结果表明,与基线模型相比本文模型在融入了上下文语境特征后取得了较明显的性能提升。

关键词: 情感分析, 自然语言理解, 准注意力机制, 上下文注意

Abstract: Aspect-based sentiment analysis aims to predict the sentiment polarity of a specific aspect in a sentence or document. Most of recent research is to use attention mechanism to model the global. But there is a problem in that the context information needs to be considered according to different contextual contexts when the BERT model is used to calculate the dependencies between representations to extract features by sentiment classification models, which leads to the lack of contextual knowledge of the modelled features. And the importance of aspect words needs to be given more attention to affect the overall classification performance of the model. To address the problems above, this paper proposes a DFLGA-BERT (Dual Features Local-Global Attention model with BERT) in response to this problem. Local and global feature extraction modules are designed respectively to fully capture the semantic association between aspect words and context. Moreover, an improved quasi-attention mechanism is used in DFLGA-BERT, which leads the model to use minus attention in the fusion of attention to weaken the effect of noise on classification in the text. The feature fusion structure of local and global features is designed to integrate better regional and international features based on CLN (Conditional Layer Normalization). Experiments were conducted on the SentiHood and SemEval 2014 Task 4 datasets. Experimental results show that the proposed model significantly improved compared to the baselines after incorporating contextual features.

Key words: Sentiment Analysis, Natural Language Understanding, Quasi-Attention Mechanism, Context Attention