融合多种类型语法信息的属性级情感分析模型

doi:10.3778/j.issn.1673-9418.2009003

计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (2): 395-402.DOI: 10.3778/j.issn.1673-9418.2009003

融合多种类型语法信息的属性级情感分析模型

肖泽管¹, 陈清亮¹^,²^,⁺()

1.暨南大学计算机科学系,广州 510632
2.云趣科技-暨南大学人工智能联合研发中心,广州 510632

收稿日期:2020-09-03 修回日期:2021-01-08 出版日期:2022-02-01 发布日期:2021-01-28
通讯作者: + E-mail: tpchen@jnu.edu.cn
作者简介:肖泽管（1995—）,男,海南万宁人,硕士研究生,主要研究方向为自然语言处理、属性级情感分析等。
陈清亮（1980—）,男,广东韶关人,博士,教授,主要研究方向为机器学习、自然语言处理等。
基金资助:
国家自然科学基金(61772232)

Aspect-Based Sentiment Analysis Model with Multiple Grammatical Information

XIAO Zeguan¹, CHEN Qingliang¹^,²^,⁺()

1. Department of Computer Science, Jinan University, Guangzhou 510632, China
2. Yunqu-Jinan University Joint AI Research Center, Guangzhou 510632, China

Received:2020-09-03 Revised:2021-01-08 Online:2022-02-01 Published:2021-01-28
About author:XIAO Zeguan, born in 1995, M.S. candidate. His research interests include natural language processing, aspect based sentiment analysis, etc.
CHEN Qingliang, born in 1980, Ph.D., professor. His research interests include machine learning, natural language processing, etc.
Supported by:
National Natural Science Foundation of China(61772232)

摘要/Abstract

摘要：

属性级情感分析（ABSA）的目标是识别出句子中属性的情感倾向。现有的方法大多使用注意力机制隐性地建模属性与上下文中情感表达的关系,而忽略了使用语法信息。一方面,属性的情感倾向与句子中的情感表达有紧密的联系,利用句子的句法结构可以更直接地对两者建模;另一方面,由于现有的基准数据集较小,模型无法充分学习通用语法知识,这使得它们难以处理复杂的句型和情感表达。针对以上问题,提出一种利用多种类型语法信息的神经网络模型。该模型采用基于依存句法树的图卷积神经网络（GCN）,并利用句法结构信息直接匹配属性与其对应情感表达,缓解冗余信息对分类的干扰。同时,使用预训练模型BERT具有多种类型的语法信息的中间层表示作为指导信息,给予模型更多的语法知识。每一层GCN的输入结合上一层GCN的输出和BERT中间层指导信息。最后将属性在最后一层GCN的表示作为特征进行情感倾向分类。通过在SemEval 2014 Task4 Restaurant、Laptop和Twitter数据集上的实验结果表明,提出模型的分类效果超越了很多基准模型。

关键词: 属性级, 情感分析, 基于变换器的双向编码器表示技术（BERT）, 依存句法树, 图卷积神经网络（GCN）

Abstract:

The aim of aspect based sentiment analysis (ABSA) is to classify the sentiment polarity towards a particular aspect in a sentence. Existing approaches usually apply attention mechanism to modeling the connection between aspects and opinion expression in an implicit way. However, they ignore the potentially useful grammatical information. On one hand, the sentiment polarity of an aspect is closely related to opinion expression, and syntactic information helps to better model the relation of them. On the other hand, models are hard to learn general grammatical knowledge when trained on existing small benchmark datasets, resulting in difficulty to handle complex sentence patterns and opinion expressions. To address the problem, this paper proposes a neural network that combines various kinds of grammatical information to enhance the accuracy. This paper employs dependency tree based graph convolutional networks (GCN) to match aspects and their corresponding opinion expression directly using syntactic information and eliminate useless information. This paper also uses middle layers of BERT as guiding information to enhance the model, which contains various kinds of contextual and grammatical information. The input of each GCN layer fuses the output of the preceding GCN layer and BERT (bidirectional encoder representations from transformers) middle layers. Finally, the aspect representation of the last layer GCN is used as the feature to identify sentiment polarity. Experiments on SemEval 2014 Task4 Restaurant, Laptop and Twitter show the model outperforms other baselines.

Key words: aspect, sentiment analysis, bidirectional encoder representations from transformers (BERT), dependency tree, graph convolutional networks (GCN)

中图分类号:

TP391.1

肖泽管, 陈清亮. 融合多种类型语法信息的属性级情感分析模型[J]. 计算机科学与探索, 2022, 16(2): 395-402.

XIAO Zeguan, CHEN Qingliang. Aspect-Based Sentiment Analysis Model with Multiple Grammatical Information[J]. Journal of Frontiers of Computer Science and Technology, 2022, 16(2): 395-402.

图/表 6

参考文献 23

[1]	LIU B. Sentiment analysis and opinion mining[M]//HIRST G. Synbook Lectures on Human Language Technologies. San Rafael: Morgan & Claypool, 2012: 1-167.
[2]	WAGNER J, ARORA P, CORTES S, et al. DCU: aspect-based polarity classification for SemEval task 4[C]//Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, Aug 23-24, 2014. Stroudsburg: ACL, 2014: 223-229.
[3]	JIN W, HO H H. A novel lexicalized HMM-based learning framework for Web opinion mining[C]//Proceedings of the 26th Annual International Conference on Machine Learning, Montreal, Jun 14-18, 2009. New York: ACM, 2009: 465-472.
[4]	RUDER S, GHAFFARI P, BRESLIN J G. A hierarchical model of reviews for aspect-based sentiment analysis[C]// Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 999-1005.
[5]	TANG D Y, QIN B, LIU T. Aspect level sentiment classification with deep memory network[C]//Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, Nov 1-4, 2016. Stroudsburg: ACL, 2016: 214-224.
[6]	MA D H, LI S J, ZHANG X D, et al. Interactive attention networks for aspect-level sentiment classification[C]//Proceedings of the 26th International Joint Conference on Artificial Intelligence, Melbourne, Aug 19-25, 2017. Menlo Park: AAAI, 2017: 4068-4074.
[7]	SONG Y W, WANG J H, JIANG T, et al. Attentional encoder network for targeted sentiment classification[J]. arXiv:1902.09314, 2019.
[8]	HUANG B X, OU Y L, CARLEY K M. Aspect level sentiment classification with attention-over-attention neural networks[C]//LNCS 10899: Proceedings of the 11th International Conference on Social, Cultural, and Behavioral Modeling,Washington, Jul 10-13, 2019. Cham: Springer, 2019: 197-206.
[9]	JAWAHAR G, SAGOT B, SEDDAH D. What does BERT learn about the structure of language?[C]//Proceedings of the 57th Conference of the Association for Computational Linguistics, Florence, Jul 28-Aug 2, 2019. Stroudsburg: ACL, 2019: 3651-3657.
[10]	DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[J]. arXiv:1810.04805, 2018.
[11]	VASWANI A, SHAZEER N, PARMAR N, et al. Attention is all you need[J]. arXiv:1706.03762, 2017.
[12]	MIKOLOV T, SUTSKEVER I, CHEN K, et al. Distributed representations of words and phrases and their compositionality[C]//Proceedings of the 27th Annual Conference on Neural Information Processing Systems, Lake Tahoe, Dec 5-8, 2013. Red Hook: Curran Associates, 2013: 3111-3119.
[13]	PENNINGTON J, SOCHER R, MANNING C D. Glove: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg:ACL, 2014: 1532-1543.
[14]	KIPF T N, WELLING M. Semi-supervised classification with graph convolutional networks[J]. arXiv:1609.02907, 2016.
[15]	KIPF T. Graph convolutional networks[EB/OL]. [2021-01-01]. https://tkipf.github.io/graph-convolutional-networks/.
[16]	LI D, WEI F R, TAN C Q, et al. Adaptive recursive neural network for target-dependent Twitter sentiment classification[C]//Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, Baltimore, Jun 22-27, 2014. Stroudsburg: ACL, 2014: 49-54.
[17]	NGUYEN T H, SHIRAI K. PhraseRNN: phrase recursive neural network for aspect-based sentiment analysis[C]//Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, Lisbon, Sep 17-21, 2015. Stroudsburg: ACL, 2015: 2509-2514.
[18]	LAKKARAJU H, SOCHER R, MANNING C J. Aspect specific sentiment analysis using hierarchical deep learning[C]//Proceedings of the 28th Annual Conference on Neural Information Processing Systems, Montreal, Dec 12, 2014. Red Hook: Curran Associates, 2014: 1-9.
[19]	HUANG B X, CARLEY K M. Syntax-aware aspect level sentiment classification with graph attention networks[C]// Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5469-5477.
[20]	SUN K, ZHANG R C, MENSAH S, et al. Aspect-level sentiment analysis via convolution over dependency tree[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5679-5688.
[21]	ZHANG C, LI Q C, SONG D W. Aspect-based sentiment classification with aspect-specific graph convolutional networks[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 4568-4578.
[22]	PONTIKI M, GALANIS D, PAVLOPOULOS J, et al. SemEval- 2014 Task 4: aspect based sentiment analysis[C]//Proceedings of the 8th International Workshop on Semantic Evaluation, Dublin, Aug 23-24, 2014. Stroudsburg: ACL, 2014: 27-35.
[23]	KINGMA D P, BA J. Adam: a method for stochastic optimization[J]. arXiv:1412.6980, 2014.

Dataset	Positive		Neutral		Negative
Dataset	Train	Test	Train	Test	Train	Test
Restaurant	2 164	728	633	196	805	196
Laptop	987	341	460	169	866	128
Twitter	1 561	173	3 127	346	1 560	173

Dataset	Positive		Neutral		Negative
Dataset	Train	Test	Train	Test	Train	Test
Restaurant	2 164	728	633	196	805	196
Laptop	987	341	460	169	866	128
Twitter	1 561	173	3 127	346	1 560	173

Method	Restaurant		Laptop		Twitter
Method	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
IAN^*	78.60	—	72.10	—	—	—
AOA^*	81.20	—	74.50	—	—	—
AEN-BERT	81.74	71.24	78.35	73.71	73.78	72.33
CDT	81.78	72.32	75.73	71.42	74.00	72.19
TD-GAT-BERT^*	83.00	—	79.80	—	—	—
ASGCN^*	80.77	72.02	75.55	71.05	72.15	70.40
ASGCN-BERT	84.67	76.95	78.98	74.89	75.08	73.68
BERT-SPC	85.29	77.67	78.02	73.10	74.38	73.01
Ours_1,5,9	85.10	77.65	78.21	74.02	75.96	74.69
Ours₁₂	85.33	77.93	78.40	74.32	76.02	74.75
Ours	85.42	79.14	78.98	75.28	76.21	74.86

Method	Restaurant		Laptop		Twitter
Method	Accuracy	Macro-F1	Accuracy	Macro-F1	Accuracy	Macro-F1
IAN^*	78.60	—	72.10	—	—	—
AOA^*	81.20	—	74.50	—	—	—
AEN-BERT	81.74	71.24	78.35	73.71	73.78	72.33
CDT	81.78	72.32	75.73	71.42	74.00	72.19
TD-GAT-BERT^*	83.00	—	79.80	—	—	—
ASGCN^*	80.77	72.02	75.55	71.05	72.15	70.40
ASGCN-BERT	84.67	76.95	78.98	74.89	75.08	73.68
BERT-SPC	85.29	77.67	78.02	73.10	74.38	73.01
Ours_1,5,9	85.10	77.65	78.21	74.02	75.96	74.69
Ours₁₂	85.33	77.93	78.40	74.32	76.02	74.75
Ours	85.42	79.14	78.98	75.28	76.21	74.86

句子	标签	AEN-BERT	Ours
While there’s decent menu, it shouldn’t take 10 minutes to get your [drinks] and 45 for a dessert pizza. 译：尽管有不错的菜单,也不应该花10分钟才上【饮料】和花45分钟才上披萨。	中立	正向	中立
Although the restaurant itself is nice, I prefer not to go for the[food]. 译：虽然这家餐馆本身很好,但我不想去吃那里的【食物】。	负向	正向	负向
The [food] was mediocre and the service was severely slow. 译：【食物】一般般,服务很慢。	中立	负向	中立
Great [food] but the service was dreadful! 译：【食物】很好,但服务很差。	正向	负向	正向
The staff members are extremely friendly and even replaced my [drink] once when I dropped it outside. 译：工作人员非常友好,甚至有一次我把【饮料】掉在外面时帮我换掉了。	中立	正向	中立
However, I can refute that [OSX] is "fast". 译：然而,我否认【OSX】“快”。	负向	正向	正向

融合多种类型语法信息的属性级情感分析模型

Aspect-Based Sentiment Analysis Model with Multiple Grammatical Information

RichHTML

PDF

可视化

摘要/Abstract

引用本文

使用本文

图/表 6

参考文献 23

相关文章 12

编辑推荐

Metrics

[1]	刘颖, 王哲, 房杰, 朱婷鸽, 李琳娜, 刘继明. 基于图文融合的多模态舆情分析[J]. 计算机科学与探索, 2022, 16(6): 1260-1278.
[2]	包广斌, 李港乐, 王国雄. 面向多模态情感分析的双模态交互注意力[J]. 计算机科学与探索, 2022, 16(4): 909-916.
[3]	李朝阳, 李琳, 陶晓辉. 面向动态交通流预测的双流图卷积网络[J]. 计算机科学与探索, 2022, 16(2): 384-394.
[4]	武家伟, 孙艳春. 融合知识图谱和深度学习方法的问诊推荐系统[J]. 计算机科学与探索, 2021, 15(8): 1432-1440.
[5]	刘继明, 张培翔, 刘颖, 张伟东, 房杰. 多模态的情感分析技术综述[J]. 计算机科学与探索, 2021, 15(7): 1165-1182.
[6]	陈虹, 杨燕, 杜圣东. 用户评论方面级情感分析研究[J]. 计算机科学与探索, 2021, 15(3): 478-485.
[7]	曹宇，李天瑞，贾真，殷成凤. BGRU:中文文本情感分析的新方法[J]. 计算机科学与探索, 2019, 13(6): 973-981.
[8]	喻涛，罗可. 利用动态多池卷积神经网络的情感分析模型[J]. 计算机科学与探索, 2018, 12(7): 1182-1190.
[9]	赵志滨，刘欢，姚兰，于戈. 中文产品评论的维度挖掘及情感分析技术研究[J]. 计算机科学与探索, 2018, 12(3): 341-349.
[10]	普次仁，侯佳林，刘月，翟东海. 深度学习算法在藏文情感分析中的应用研究[J]. 计算机科学与探索, 2017, 11(7): 1122-1130.
[11]	李天辰，殷建平. 基于主题聚类的情感极性判别方法[J]. 计算机科学与探索, 2016, 10(7): 989-994.
[12]	欧高炎，陈薇，王腾蛟，雷凯，杨冬青. 面向微博的情感影响最大化模型[J]. 计算机科学与探索, 2012, 6(9): 769-778.