Span-Level Dual-Encoder Model for Aspect Sentiment Triplet Extraction

doi:10.3778/j.issn.1673-9418.2208102

Abstract

Abstract: Aspect sentiment triplet extraction (ASTE) is one of the subtasks of aspect-based sentiment analysis, which aims to identify all aspect terms, their corresponding opinion terms and sentiment polarities in sentences. Currently, pipeline or end-to-end models are adopted to accomplish the ASTE task. The former cannot solve the overlapping problem of aspect terms in triplets and ignores the dependency between opinion terms and sentiment polarities. The latter divides the ASTE task into two subtasks of aspect-opinion-extraction and sentiment-polarity-classification, which applies multi-task learning through a shared encoder. However, this setting does not distinguish the differences between the features of the two subtasks, leading to the feature confusion problem. SD-ASTE (span-level dual-encoder model for ASTE), a pipeline model with two modules, is proposed to address the above problems. The first module extracts aspect terms and opinion terms based on spans. The span feature representation incor-porates span head, tail and length information to focus on the boundary information of aspect terms and opinion terms. The second module judges the sentiment polarities expressed by aspect-opinion span pairs. The span-pair feature representation is based on levitated markers to focus on the dependency among triplet elements. The model utilizes two independent encoders to extract different features for each module. Comparative experimental results on multiple datasets show that the model is superior to the state-of-the-art pipeline and end-to-end models. Validity experiments show the effectiveness of the span feature representation, span-pair feature representation and the two independent encoders.

Key words: sentiment analysis, aspect sentiment triplet extraction (ASTE), pipeline model, span, independent encoders

摘要： 方面情感三元组抽取（ASTE）是方面级情感分析的子任务之一，旨在识别出句子中所有的方面词及其对应的观点词和情感极性。目前，ASTE任务通过流水线模型或端到端模型完成，前者无法解决三元组方面词重叠问题，且忽视了观点词和情感极性之间的依赖关系；后者将ASTE任务分解为方面词和观点词抽取子任务以及情感极性分类子任务，通过共享编码器进行多任务学习，未区分两个子任务的特征差异，导致特征混淆问题。针对上述问题，提出了片段级别的双编码器方面情感三元组抽取模型（SD-ASTE）。该模型是流水线模型，分为两个模块。第一个模块基于片段抽取方面词和观点词，在片段特征表示中融入片段首尾和长度信息，关注方面词和观点词的边界信息；第二个模块判断方面词-观点词片段对表达的情感极性，采用基于悬浮标记的片段对特征表示方式，侧重于学习三元组各元素之间的依赖关系。模型利用两个独立编码器，分别为两模块提取不同的特征信息。多个数据集上的对比实验结果表明，该模型相较于目前最优的流水线模型和端到端模型具有更优的效果。通过有效性实验，验证了片段特征表示和片段对特征表示以及两个独立编码器的有效性。

关键词: 情感分析, 方面情感三元组抽取（ASTE）, 流水线模型, 片段, 独立编码器

ZHANG Yunqi, LI Songda, LAN Yuquan, LI Dongxu, ZHAO Hui. Span-Level Dual-Encoder Model for Aspect Sentiment Triplet Extraction[J]. Journal of Frontiers of Computer Science and Technology, 2023, 17(12): 3010-3019.

张韵琪, 李松达, 兰于权, 李东旭, 赵慧. 片段级别的双编码器方面情感三元组抽取模型[J]. 计算机科学与探索, 2023, 17(12): 3010-3019.

References

[1] ZHANG W, LI X, DENG Y, et al. A survey on aspect-based sentiment analysis: tasks, methods, and challenges[J]. arXiv: 2203.01054, 2022.
[2] PENG H, XU L, BING L, et al. Knowing what, how and why: a near complete solution for aspect-based sentiment analysis[C]//Proceedings of the 34th AAAI Conference on Artificial Intelligence, the 32nd Innovative Applications of Artificial Intelligence Conference, the 10th AAAI Sympo-sium on Educational Advances in Artificial Intelligence,New York, Feb 7-12, 2020. Menlo Park: AAAI, 2020: 8600-8607.
[3] MUKHERJEE R, NAYAK T, BUTALA Y, et al. PASTE: a tagging-free decoding framework using pointer networks for aspect sentiment triplet extraction[C]//Proceedings of the 2021 Conference on Empirical Methods in Natural Language Processing, Punta Cana, Nov 7-11, 2021. Stroudsburg: ACL, 2021: 9279-9291.
[4] XU L, CHIA Y K, BING L. Learning span-level interac-tions for aspect sentiment triplet extraction[C]//Proceedings of the 59th Annual Meeting of the Association for Computa-tional Linguistics, Aug 1-6, 2021. Stroudsburg: ACL, 2021: 4755-4766.
[5] WANG J, LU W. Two are better than one: joint entity and relation extraction with table-sequence encoders[C]//Pro-ceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20, 2020. Strouds-burg: ACL, 2020: 1706-1721.
[6] ZHANG Y, YANG Q. A survey on multi-task learning[J]. IEEE Transactions on Knowledge and Data Engineering, 2021.
[7] XU L, LI H, LU W, et al. Position-aware tagging for aspect sentiment triplet extraction[C]//Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 2339-2349.
[8] WU Z, YING C, ZHAO F, et al. Grid tagging scheme for aspect-oriented fine-grained opinion extraction[C]//Findings of the Association for Computational Linguistics: EMNLP 2020, Nov 16-20, 2020. Stroudsburg: ACL, 2020: 2576-2585.
[9] LUAN Y, WADDEN D, HE L, et al. A general framework for information extraction using dynamic span graphs[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics, Minneapolis, Jun 2-7, 2019. Stroudsburg: ACL, 2019: 3036-3046.
[10] WADDEN D, WENNBERG U, LUAN Y, et al. Entity, relation, and event extraction with contextualized span representations[C]//Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing, Hong Kong, China, Nov 3-7, 2019. Stroudsburg: ACL, 2019: 5784-5789.
[11] CHEN Y, ZHANG Y, HU C, et al. Jointly extracting explicit and implicit relational triples with reasoning pattern enhan-ced binary pointer network[C]//Proceedings of the 2021 Conference of the North American Chapter of the Associa-tion for Computational Linguistics, Jun 6-11, 2021. Strouds-burg: ACL, 2021: 5694-5703.
[12] CHEN S, WANG Y, LIU J, et al. Bidirectional machine reading comprehension for aspect sentiment triplet extrac-tion[C]//Proceedings of the 35th AAAI Conference on Artificial Intelligence, the 33rd Conference on Innovative Applications of Artificial Intelligence, the 11th Symposium on Educational Advances in Artificial Intelligence, Feb 2-9, 2021. Menlo Park: AAAI, 2021: 12666-12674.
[13] MAO Y, SHEN Y, YU C, et al. A joint training Dual-MRC framework for aspect based sentiment analysis[C]//Procee-dings of the 35th AAAI Conference on Artificial Intellige-nce, the 33rd Conference on Innovative Applications of Artificial Intelligence, the 11th Symposium on Educational Advances in Artificial Intelligence, Feb 2-9, 2021. Menlo Park: AAAI, 2021: 13543-13551.
[14] YAN H, DAI J, JI T, et al. A unified generative framework for aspect-based sentiment analysis[C]//Proceedings of the 59th Annual Meeting of the Association for Computational Linguistics, Aug 1-6, 2021. Stroudsburg: ACL, 2021: 2416-2429.
[15] LU Y, LIU Q, DAI D, et al. Unified structure generation for universal information extraction[C]//Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics, Dublin, May 22-27, 2022. Stroudsburg: ACL, 2022: 5755-5772.
[16] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[C]//Proceedings of the 2019 Conference of the North American Chapter of the Association for Com-putational Linguistics, Minneapolis, Jun 2-7, 2019. Strouds-burg: ACL, 2019: 4171-4186.
[17] ZHONG Z, CHEN D. A frustratingly easy approach for entity and relation extraction[C]//Proceedings of the 2021 Conference of the North American Chapter of the Associa-tion for Computational Linguistics, Jun 6-11, 2021. Strouds-burg: ACL, 2021: 50-61.
[18] 姚婉薇. 基于Target-Aspect-Opinion联合抽取的汽车评论情感分析[D]. 上海: 华东师范大学, 2020.
YAO W W. Joint extraction of Target-Aspect-Opinion based sentiment analysis with automobile review data[D]. Shang-hai: East China Normal University, 2020.
[19] PENNINGTON J, SOCHER R, MANNING C. GloVe: global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing, Doha, Oct 25-29, 2014. Stroudsburg:ACL, 2014: 1532-1543.
[20] SHAO Y, GENG Z, LIU Y, et al. CPT: a pre-trained unbalan-ced transformer for both Chinese language understanding and generation[J]. arXiv:2109.05729, 2021.