关键词: 多模态方面级情感分析, 双粒度图像信息, 多模态交互, 多模态融合, 跨模态注意力

Abstract: Multimodal aspect-based Sentiment Analysis (Multimodal Aspect-Based Sentiment Analysis ,MABSA), as a fine-grained sentiment analysis technique, aims to improve the accuracy and effectiveness of the field by integrating feature data from multiple modes. Most of the existing researches on multimodal aspect-based sentiment analysis focus on the cross-modal alignment between text and image modes, ignoring the potential contribution of image coarse-grained feature information to MABSA subtasks. Therefore, a multimodal aspect-based sentiment analysis method combining two granularity image information(Combining Two Granularity Image Information for Multi-Modal Aspect-Based Sentiment Analysis ,CTGI) is proposed in this paper. Specifically, in the multi-modal aspect term extraction task, in order to enhance the interaction between image and text modes, ClipCap is used to obtain the coarse-grained feature description text of the image, which is used as image prompt information to assist the model to predict the aspect terms and their attributes in the text. In terms of multi-modal emotion classification, in order to capture rich fine-grained emotional features of images, the cross-modal attention mechanism is used to interact the underlying features of images with original emotional semantics with the masked text through multiple layers of depth, so as to strengthen the fusion of image features into text features. Experimental results on two public Twitter datasets and Restaurant+ datasets show that CTGI performs better than the current baseline model, which validates the rationality of different contribution degrees of image coarse-granularity to MABSA subtasks.

Key words: Multi-modal Aspect-based Sentiment Analysis, Two Granularity Image Information, Multi-modal interaction, Multi-modal fusion, Cross-modal Attention