计算机科学与探索 ›› 2022, Vol. 16 ›› Issue (6): 1260-1278.DOI: 10.3778/j.issn.1673-9418.2110056

• 综述·探索 • 上一篇    下一篇

基于图文融合的多模态舆情分析

刘颖1,2,+(), 王哲1, 房杰1,2, 朱婷鸽1,2, 李琳娜3, 刘继明4   

  1. 1. 西安邮电大学 图像与信息处理研究所,西安 710121
    2. 西安邮电大学 电子信息现场勘验应用技术公安部重点实验室,西安 710121
    3. 西安邮电大学 网络舆情监测与分析中心,西安 710121
    4. 西安邮电大学 通信与信息工程学院,西安 710121
  • 收稿日期:2021-10-22 修回日期:2022-01-12 出版日期:2022-06-01 发布日期:2022-06-20
  • 通讯作者: + E-mail: liuying_ciip@163.com
  • 作者简介:刘颖(1972—),女,陕西户县人,博士,教授,硕士生导师,电子信息现场勘验应用技术公安部重点实验室总工程师,主要研究方向为图像检索、图像清晰化等。
    王哲(1994—),男,辽宁鞍山人,硕士研究生,主要研究方向为基于图文融合的多模态舆情分析。
    房杰(1993—),男,陕西咸阳人,博士,副教授,主要研究方向为视觉影像的语义理解及其应用。
    朱婷鸽(1976—),女,陕西杨凌人,博士,主要研究方向为图像信息安全。
    李琳娜(1980—),女,陕西西安人,西安邮电大学网络舆情监测与分析中心主任,主要研究方向为网络舆情分析。
    刘继明(1964—),男,福建龙岩人,博士,网经科技(苏州)有限公司董事长,西安邮电大学特聘教授,主要研究方向为人工智能技术及其产业化。
  • 基金资助:
    公安部科技强警项目(2019GABJC41)

Multi-modal Public Opinion Analysis Based on Image and Text Fusion

LIU Ying1,2,+(), WANG Zhe1, FANG Jie1,2, ZHU Tingge1,2, LI Linna3, LIU Jiming4   

  1. 1. Center for Image and Information Processing, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
    2. Key Laboratory of Electronic Information Application Technology for Crime Scene Investigation, Ministry of Public Security, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
    3. Network Public Opinion Monitoring and Analysis Center, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
    4. School of Communications and Information Engineering, Xi’an University of Posts and Telecommunications, Xi’an 710121, China
  • Received:2021-10-22 Revised:2022-01-12 Online:2022-06-01 Published:2022-06-20
  • About author:LIU Ying, born in 1972, Ph.D., professor, M.S. supervisor. Her research interests include image retrieval, image clarity, etc.
    WANG Zhe, born in 1994, M.S. candidate. His research interest is multi-modal public opinion analysis based on image and text fusion.
    FANG Jie, born in 1993, Ph.D., associate professor. His research interest is semantic understanding of visual image and its application.
    ZHU Tingge, born in 1976, Ph.D. Her research interest is image information security.
    LI Linna, born in 1980, director of Network Public Opinion Monitoring and Analysis Center of Xi’an University of Posts and Telecommunications. Her research interest is network public opinion analysis.
    LIU Jiming, born in 1964, Ph.D., distinguished professor at Xi’an University of Posts and Telecommunications. His research interests include artificial intelligence technology and its industrialization.
  • Supported by:
    Science and Technology Project under Ministry of Public Security of China(2019GABJC41)

摘要:

由于互联网以及移动手机的不断普及,人们逐渐进入到一个参与式的网络时代,越来越多的人们喜欢在网络上通过文本和图像的方式发布自己的观点、评论以及情感。对于这些文本和图像信息进行有效分析,不仅可以帮助企业更好地提高产品的质量,而且有利于为政府决策和社会生产生活提供指导。对基于多模态图文融合的网络舆情情感分析进行了综述。首先对舆情分析的基本概念进行了概括;其次对社交媒体上单模态的文本和视觉舆情情感分析的过程进行了说明;然后对基于图文融合的舆情分析算法进行了总结,并按照不同融合策略,将其分为特征层融合、决策层融合和线性回归模型;另外总结了针对社交媒体的多模态情感分析的常用数据集;最后讨论了网络舆情分析的难点以及未来研究方向。

关键词: 网络舆情分析, 图文融合, 情感分析, 多模态

Abstract:

Due to the continuous popularization of the Internet and mobile phones, people have gradually entered a participatory network era. More and more people like to publish their opinions, comments and emotions through text and image on the Internet. Effective analysis of these text and image information can not only help companies better improve the quality of their products, but also provide guidance for government decision-making and social production and life. This paper summarizes the sentiment analysis of online public opinion based on multi-modal image and text fusion. Firstly, it summarizes the basic concepts of public opinion analysis. Secondly, it explains the process of single-modal text and visual sentiment analysis on social media. Thirdly, it summarizes the public opinion analysis algorithms based on image and text fusion, and divides the algorithms into feature layer fusion, decision layer fusion and linear regression model according to different fusion strategies. In addition, it summarizes the commonly used multi-modal sentiment analysis for social media dataset. Finally, the difficulties of online opinion analysis and future research directions are discussed.

Key words: network public opinion analysis, image and text fusion, sentiment analysis, multi-modal

中图分类号: