Local and Global Feature Fusion Network Model for Aspect-Based Sentiment Analysis

doi:10.3778/j.issn.1673-9418.2107069

Abstract

Abstract: Aspect-based sentiment analysis aims to predict the sentiment polarity of a specific aspect in a sentence or document. Most of the recent research is to use attention mechanism and external semantic knowledge to model the global context. The sentiment polarity of an aspect often depends on the local context which is highly related to the aspect, but most models focus too much on the global context, which makes the parameter amount of the model generally larger, resulting in an increase in the amount of calculation. To this end, this paper proposes a lightweight network model based on multi-head attention mechanism, which is named local and global feature fusion network. Firstly, a bidirectional gated recurrent unit is used to encode the context. Secondly, context words that are less relevant to the aspect term are masked out according to the semantic relation distance with the aspect term, so as to obtain the local context representation. Finally, the local and global context are extracted respectively by multi-head Aspect-aware attention network, and the results of the two extraction are combined. In addition, the pre-trained BERT is also applied to the proposed model and better results are obtained. Experiments are conducted on three datasets: Twitter, Laptop, and Restaurant. Accuracy and F1 indicators are used for evaluation. Experimental results show that the proposed model achieves better results than other aspect-based sentiment classification algorithms with a small amount of parameters.

Key words: attention mechanism, local context attention, sentiment analysis, text classification

摘要： 方面情感分析旨在预测句子或文档中一个特定方面的情感极性。最近大部分的研究都是通过使用注意力机制及外部语义知识对全局上下文进行建模，以完成这项工作。方面的情感极性往往取决于与方面高度相关的局部上下文，但大多数模型将过多的注意力集中于全局上下文，这使得模型的参数量普遍比较大，导致计算量也随之增大。为此，提出一种基于多头注意力机制的轻量化网络模型——局部与全局特征融合网络模型。首先，使用双向门控循环单元来对上下文进行编码。其次，根据与方面项的语义相关距离掩蔽掉与方面项相关度较小的上下文词，以此得到局部上下文表示。最后，通过多头Aspect-aware注意力网络对局部和全局上下文分别进行提取，将两者提取的结果进行结合。此外，还将预先训练的BERT应用于这项任务，并获得了更好的结果。在三个数据集Twitter、Laptop、Restaurant上进行实验，采用Accuracy和[F1]指标进行评估。实验结果表明，该模型在参数量较小的情况下，取得了比其他基于方面的情感分类算法更好的结果。

关键词: 注意力机制, 局部上下文注意, 情感分析, 文本分类

XIA Hongbin, LI Qiang, LIU Yuan. Local and Global Feature Fusion Network Model for Aspect-Based Sentiment Analysis[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(4): 902-911.

夏鸿斌, 李强, 刘渊. 局部与全局特征融合的方面情感分析网络模型[J]. 计算机科学与探索, 2023, 17(4): 902-911.

References

[1] YADAV A, VISHWAKARMA D K. Sentiment analysis using deep learning architectures: a review[J]. Artificial Intelligence Review, 2020, 53(6): 4335-4385.
[2] RUDER S, GHAFFARI P, BRESLIN J G. A hierarchical model of reviews for aspect-based sentiment analysis[C]//Procee-dings of the 2016 Conference on Empirical Methods in Na-tural Language Processing, Austin, Nov 1-4, 2016. Strouds-burg: ACL, 2016: 999-1005.
[3] TANG D, QIN B, FENG X, et al. Effective LSTMs for target-dependent sentiment classification[C]//Proceedings of the 26th International Conference on Computational Linguistics, Osaka, Dec 11-16, 2016. Stroudsburg: ACL, 2016: 3298-3307.
[4] TANG D, QIN B, LIU T. Aspect level sentiment classifica-tion with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Pro-cessing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 214-224.
[5] WANG S, MAZUMDER S, LIU B, et al. Target-sensitive memory networks for aspect sentiment classification[C]// Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, Melbourne, Jul 15-20, 2018.Stroudsburg: ACL, 2018: 957-967.
[6] LIU J, ZHANG Y. Attention modeling for targeted senti-ment[C]//Proceedings of the 15th Conference of the Euro-pean Chapter of the Association for Computational Lin-guistics, Valencia, Apr 3-8, 2017. Stroudsburg: ACL, 2017: 572-577.
[7] ZHANG Y, QI P, MANNING C D. Graph convolution over pruned dependency trees improves relation extraction[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Oct 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 2205-2215.
[8] SUN K, ZHANG R, MENSAH S, et al. Aspect-level senti-ment analysis via convolution over dependency tree[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5679-5688.
[9] HUANG B, CARLEY K. Syntax-aware aspect level senti-ment classification with graph attention networks[C]//Procee-dings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5469-5477.
[10] ZENG B, YANG H, XU R, et al. LCF: a local context focus mechanism for aspect-based sentiment classification[J]. App-lied Sciences, 2019, 9(16): 3389.
[11] PHAN M H, OGUNBONA P O. Modelling context and syntactical features for aspect-based sentiment analysis[C]//Proceedings of the 58th Annual Meeting of the Asso-ciation for Computational Linguistics, Jul 5-10, 2020. Strouds-burg: ACL, 2020: 3211-3220.
[12] WANG Y, HUANG M, ZHAO L. Attention-based LSTM for aspect-level sentiment classification[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Lan-guage Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 606-615.
[13] MA D, LI S, ZHANG X, et al. Interactive attention net-works for aspect-level sentiment classification[C]//Procee-dings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Aug 19-25, 2017: 4068-4074.
[14] FAN F F, FENG Y S, ZHAO D Y. Multi-grained attention network for aspect-level sentiment classification[C]//Pro-ceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, Brussels, Octr 31-Nov 4, 2018. Stroudsburg: ACL, 2018: 3433-3442.
[15] ZHAO F, WU Z, DAI X. Attention transfer network for aspect-level sentiment classification[C]//Proceedings of the 28th International Conference on Computational Linguistics, Barcelona, Jul 8-13, 2020. Stroudsburg: ACL, 2020: 811-821.
[16] XU H, LIU B, SHU L, et al. BERT post-training for review reading comprehension and aspect-based sentiment analysis[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Minneapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 2324-2335.
[17] DEVLIN J, CHANG M, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understand-ing[J]. arXiv:1810.04805, 2018.
[18] SONG Y W, WANG J H, JIANG T, et al. Attentional encoder network for targeted sentiment classification[J]. arXiv:1902.09314, 2019.
[19] LI X L, FU X Y, XU G L, et al. Enhancing BERT repre-sentation with context-aware embedding for aspect-based sentiment analysis[J]. IEEE Access, 2020, 8: 46868-46876.
[20] GAO Z J, FENG A, SONG X Y, et al. Target-dependent sentiment classification with BERT[J]. IEEE Access, 2019, 7: 154290-154299.
[21] ZHANG C, LI Q， SONG D. Aspect-based sentiment classi-fication with aspect-specific graph convolutional networks[C]//Proceedings of the 2019 Conference on Empirical Me-thods in Natural Language Processing and the 9th Inter-national Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 4567-4577.
[22] LIU P， BAI X， ZHANG Y. Investigating typed syntactic dependencies for targeted sentiment classification using graph attention neural network[J]. arXiv:2002.09685, 2020.
[23] WANG K, SHEN W, YANG Y, et al. Relational graph at-tention network for aspect-based sentiment analysis[C]//Pro-ceedings of the 58th Annual Meeting of the Association for Computational Linguistics, Jul 5-10, 2020. Stroudsburg: ACL, 2020: 3229-3238.
[24] LIANG B, YIN R, GUI L, et al. Jointly learning aspect-focused and inter-aspect relations with graph convolutional networks for aspect sentiment analysis[C]//Proceedings of the 28th International Conference on Computational Lin-guistics, Barcelona, Dec 8-13, 2020: 150-161.
[25] PENNINGTON J, SOCHER R, MANNING C. GloVe: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg: ACL, 2014: 1532-1543.
[26] DONG L, WEI F R, TAN C Q, el al. Adaptive recursive neural network for target-dependent Twitter sentiment classi-fication[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Jun 22-27, 2014. Stroudsburg: ACL, 2014: 49-54.
[27] CHEN P, SUN Z Q, BING L D, et al. Recurrent attention network on memory for aspect sentiment analysis[C]//Pro-ceedings of the 2017 Conference on Empirical Methods in Natural Language Processing, Copenhagen, Sep 9-11, 2017.Stroudsburg: ACL, 2017: 452-461.
[28] LIU N, SHEN B. Aspect-based sentiment analysis with gated alternate neural network[J]. Knowledge-Based Systems, 2019, 188: 105010-105024.
[29] ZHANG M, QIAN T Y. Convolution over hierarchical syn-tactic and lexical graphs for aspect level sentiment analysis[C]//Proceedings of the 2020 Conference on Empirical Me-thods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 3540-3549.
[30] LI R F, CHEN H, FENG F X, et al. Dual graph convolu-tional networks for aspect-based sentiment analysis[C]//Pro-ceedings of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing, Aug 1-6, 2021. Stroudsburg: ACL, 2021: 6319-6329.