计算机科学与探索

• 学术研究 •    下一篇

融合图文多粒度情感特征的多模态谣言检测方法

刘先博,向澳,杜彦辉   

  1. 1. 中国人民公安大学 信息网络安全学院,北京 100038
    2. 武汉大学 计算机学院,武汉430072

Multimodal rumor detection method based on multi-granularity emotional features of image-text

LIU Xianbo,  XIANG Ao,  DU Yanhui   

  1. 1. College of Information Network Security, People 's Public Security University of China, Beijing 100038, China
    2. School of Computer Science, Wuhan University, Wuhan 430072, China

摘要: 涉及公共安全、灾害事故等群体性事件谣言往往在文字或图像中含有丰富的情感特征信息,极易调动网民情绪反应,诱导其点赞、评论和转发。然而,现有的多模态谣言检测方法对于多模态数据中蕴含的情感特征缺乏有效的提取方法,并且在特征融合过程中没有考虑模态间的关系,存在一定冗余特征的问题。为探究跨模态情感特征在谣言检测中的作用,提出一种融合图文多粒度情感特征的多模态谣言检测方法,该方法在不依赖评论、传播模式等社会信息的前提下,将图文多粒度情感特征融入到多模态谣言检测方法中,利用基于交互注意力机制的跨模态多粒度情感特征融合方法充分融合多媒体信息的深层特征,并在Weibo和Twitter两个公开数据集进行对比实验和消融实验。结果表明,该方法与现有谣言检测方法相比,在两个数据集中的谣言检测准确率分别提升到91.2%和83.9%,在F1值等多个指标上展现了优异性能,有效提升了谣言检测性能和模型的可解释性,一定程度上能够辅助公安机关开展群体性事件的谣言处置工作,为基层警务实战提供技术支持。

关键词: 谣言检测, 情感分析, 多模态融合, 注意力机制, 群体性事件

Abstract: Rumors involving public safety, disasters, and other mass incidents often contain rich emotional features in text or images, which easily mobilize netizens' emotional responses, inducing them to like, comment, and share. However, existing multimodal rumor detection methods lack effective extraction techniques for the emotional features contained in multimodal data and fail to consider the interrelationship between modalities during feature fusion, resulting in redundant and less accurate feature representations. To explore the role of cross-modal emotional features in rumor detection, a multimodal rumor detection method that integrates multi-granularity emotional features of image-text is proposed. Without relying on social information such as comments and dissemination patterns, this method integrates multi-granularity emotional features into the multimodal rumor detection process. It employs a cross-modal multi-granularity emotional feature fusion method based on an interactive attention mechanism to fully integrate deep features of multimedia information. To evaluate the effectiveness of the proposed method, comparative and ablation experiments were conducted on two public datasets. The results indicate that the proposed method improves rumor detection accuracy to 91.2% on the Weibo dataset and 83.9% on the Twitter dataset, showing superior performance across multiple metrics such as F1 value , effectively enhancing rumor detection performance and the interpretability of the model. To some extent, it can assist public security agencies in handling rumors during mass incidents, providing technical support for grassroots police operations.

Key words: rumor detection , emotional analysis , multimodal fusion , attention mechanism , mass incidents