Aspect-Based Sentiment Analysis Model with Multiple Grammatical Information

doi:10.3778/j.issn.1673-9418.2009003

Abstract

Abstract:

The aim of aspect based sentiment analysis (ABSA) is to classify the sentiment polarity towards a particular aspect in a sentence. Existing approaches usually apply attention mechanism to modeling the connection between aspects and opinion expression in an implicit way. However, they ignore the potentially useful grammatical information. On one hand, the sentiment polarity of an aspect is closely related to opinion expression, and syntactic information helps to better model the relation of them. On the other hand, models are hard to learn general grammatical knowledge when trained on existing small benchmark datasets, resulting in difficulty to handle complex sentence patterns and opinion expressions. To address the problem, this paper proposes a neural network that combines various kinds of grammatical information to enhance the accuracy. This paper employs dependency tree based graph convolutional networks (GCN) to match aspects and their corresponding opinion expression directly using syntactic information and eliminate useless information. This paper also uses middle layers of BERT as guiding information to enhance the model, which contains various kinds of contextual and grammatical information. The input of each GCN layer fuses the output of the preceding GCN layer and BERT (bidirectional encoder representations from transformers) middle layers. Finally, the aspect representation of the last layer GCN is used as the feature to identify sentiment polarity. Experiments on SemEval 2014 Task4 Restaurant, Laptop and Twitter show the model outperforms other baselines.

Key words: aspect, sentiment analysis, bidirectional encoder representations from transformers (BERT), dependency tree, graph convolutional networks (GCN)

摘要：

属性级情感分析（ABSA）的目标是识别出句子中属性的情感倾向。现有的方法大多使用注意力机制隐性地建模属性与上下文中情感表达的关系,而忽略了使用语法信息。一方面,属性的情感倾向与句子中的情感表达有紧密的联系,利用句子的句法结构可以更直接地对两者建模;另一方面,由于现有的基准数据集较小,模型无法充分学习通用语法知识,这使得它们难以处理复杂的句型和情感表达。针对以上问题,提出一种利用多种类型语法信息的神经网络模型。该模型采用基于依存句法树的图卷积神经网络（GCN）,并利用句法结构信息直接匹配属性与其对应情感表达,缓解冗余信息对分类的干扰。同时,使用预训练模型BERT具有多种类型的语法信息的中间层表示作为指导信息,给予模型更多的语法知识。每一层GCN的输入结合上一层GCN的输出和BERT中间层指导信息。最后将属性在最后一层GCN的表示作为特征进行情感倾向分类。通过在SemEval 2014 Task4 Restaurant、Laptop和Twitter数据集上的实验结果表明,提出模型的分类效果超越了很多基准模型。

关键词: 属性级, 情感分析, 基于变换器的双向编码器表示技术（BERT）, 依存句法树, 图卷积神经网络（GCN）

CLC Number:

TP391.1

XIAO Zeguan, CHEN Qingliang. Aspect-Based Sentiment Analysis Model with Multiple Grammatical Information[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 395-402.

肖泽管, 陈清亮. 融合多种类型语法信息的属性级情感分析模型[J]. 计算机科学与探索, 2022, 16(2): 395-402.

Figures/Tables 6

References 23

[1]	LIU B. Sentiment analysis and opinion mining[M]//HIRST G. Synbook Lectures on Human Language Technologies. San Rafael: Morgan & Claypool, 2012: 1-167.
[2]	WAGNER J, ARORA P, CORTES S, et al. DCU: aspect-based polarity classification for SemEval task 4[C]//Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, Aug 23-24, 2014. Stroudsburg: ACL, 2014: 223-229.
[3]	JIN W, HO H H. A novel lexicalized HMM-based learning framework for Web opinion mining[C]//Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Jun 14-18, 2009. New York: ACM, 2009: 465-472.
[4]	RUDER S, GHAFFARI P, BRESLIN J G. A hierarchical model of reviews for aspect-based sentiment analysis[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 999-1005.
[5]	TANG D Y, QIN B, LIU T. Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 214-224.
[6]	MA D H, LI S J, ZHANG X D, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Aug 19-25, 2017. Menlo Park: AAAI, 2017: 4068-4074.
[7]	SONG Y W, WANG J H, JIANG T, et al. Attentional encoder network for targeted sentiment classification[J]. arXiv:1902.09314, 2019.
[8]	HUANG B X, OU Y L, CARLEY K M. Aspect level sentiment classification with attention-over-attention neural networks[C]//LNCS 10899: Proceedings of the 11th International Conference on Social, Cultural, and Behavioral Modeling,Washington, Jul 10-13, 2019. Cham: Springer, 2019: 197-206.
[9]	JAWAHAR G, SAGOT B, SEDDAH D. What does BERT learn about the structure of language?[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 3651-3657.
[10]	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[11]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. arXiv:1706.03762, 2017.
[12]	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 3111-3119.
[13]	PENNINGTON J, SOCHER R, MANNING C D. Glove: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg:ACL, 2014: 1532-1543.
[14]	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[15]	KIPF T. Graph convolutional networks[EB/OL]. [2021-01-01]. https://tkipf.github.io/graph-convolutional-networks/.
[16]	LI D, WEI F R, TAN C Q, et al. Adaptive recursive neural network for target-dependent Twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Jun 22-27, 2014. Stroudsburg: ACL, 2014: 49-54.
[17]	NGUYEN T H, SHIRAI K. PhraseRNN: phrase recursive neural network for aspect-based sentiment analysis[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Stroudsburg: ACL, 2015: 2509-2514.
[18]	LAKKARAJU H, SOCHER R, MANNING C J. Aspect specific sentiment analysis using hierarchical deep learning[C]//Proceedings of the 28th Annual Conference on Neural Information Processing Systems, Montreal, Dec 12, 2014. Red Hook: Curran Associates, 2014: 1-9.
[19]	HUANG B X, CARLEY K M. Syntax-aware aspect level sentiment classification with graph attention networks[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5469-5477.
[20]	SUN K, ZHANG R C, MENSAH S, et al. Aspect-level sentiment analysis via convolution over dependency tree[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5679-5688.
[21]	ZHANG C, LI Q C, SONG D W. Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 4568-4578.
[22]	PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval- 2014 Task 4: aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, Aug 23-24, 2014. Stroudsburg: ACL, 2014: 27-35.
[23]	KINGMA D P, BA J. Adam: a method for stochastic optimization[J]. arXiv:1412.6980, 2014.

Dataset	Positive		Neutral		Negative
Dataset	Train	Test	Train	Test	Train	Test
Restaurant	2 164	728	633	196	805	196
Laptop	987	341	460	169	866	128
Twitter	1 561	173	3 127	346	1 560	173

Dataset	Positive		Neutral		Negative
Dataset	Train	Test	Train	Test	Train	Test
Restaurant	2 164	728	633	196	805	196
Laptop	987	341	460	169	866	128
Twitter	1 561	173	3 127	346	1 560	173

Method	Restaurant		Laptop		Twitter
Method	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
IAN^*	78.60	—	72.10	—	—	—
AOA^*	81.20	—	74.50	—	—	—
AEN-BERT	81.74	71.24	78.35	73.71	73.78	72.33
CDT	81.78	72.32	75.73	71.42	74.00	72.19
TD-GAT-BERT^*	83.00	—	79.80	—	—	—
ASGCN^*	80.77	72.02	75.55	71.05	72.15	70.40
ASGCN-BERT	84.67	76.95	78.98	74.89	75.08	73.68
BERT-SPC	85.29	77.67	78.02	73.10	74.38	73.01
Ours_1,5,9	85.10	77.65	78.21	74.02	75.96	74.69
Ours₁₂	85.33	77.93	78.40	74.32	76.02	74.75
Ours	85.42	79.14	78.98	75.28	76.21	74.86

Method	Restaurant		Laptop		Twitter
Method	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
IAN^*	78.60	—	72.10	—	—	—
AOA^*	81.20	—	74.50	—	—	—
AEN-BERT	81.74	71.24	78.35	73.71	73.78	72.33
CDT	81.78	72.32	75.73	71.42	74.00	72.19
TD-GAT-BERT^*	83.00	—	79.80	—	—	—
ASGCN^*	80.77	72.02	75.55	71.05	72.15	70.40
ASGCN-BERT	84.67	76.95	78.98	74.89	75.08	73.68
BERT-SPC	85.29	77.67	78.02	73.10	74.38	73.01
Ours_1,5,9	85.10	77.65	78.21	74.02	75.96	74.69
Ours₁₂	85.33	77.93	78.40	74.32	76.02	74.75
Ours	85.42	79.14	78.98	75.28	76.21	74.86

句子	标签	AEN-BERT	Ours
While there’s decent menu, it shouldn’t take 10 minutes to get your [drinks] and 45 for a dessert pizza. 译：尽管有不错的菜单,也不应该花10分钟才上【饮料】和花45分钟才上披萨。	中立	正向	中立
Although the restaurant itself is nice, I prefer not to go for the[food]. 译：虽然这家餐馆本身很好,但我不想去吃那里的【食物】。	负向	正向	负向
The [food] was mediocre and the service was severely slow. 译：【食物】一般般,服务很慢。	中立	负向	中立
Great [food] but the service was dreadful! 译：【食物】很好,但服务很差。	正向	负向	正向
The staff members are extremely friendly and even replaced my [drink] once when I dropped it outside. 译：工作人员非常友好,甚至有一次我把【饮料】掉在外面时帮我换掉了。	中立	正向	中立
However, I can refute that [OSX] is "fast". 译：然而,我否认【OSX】“快”。	负向	正向	正向