关键词: 时域卷积网络, 特征融合, 多模态情感分析, 注意力机制

Abstract: Aiming at the problem that traditional sentiment analysis methods cannot solve the problem of short video sentiment expression and the problem that existing multimodal sentiment analysis methods have low accuracy and poor interaction between different modal information, a multimodal sentiment analysis model with composite hierarchical fusion is established by studying multimodal sentiment analysis methods and combining temporal convolutional network (TCN) and Soft-attention mechanism. The model first dimensionally equalizes the text features, video facial features and audio features extracted from the video, and then fuses the obtained information features between different modalities in a composite manner (The model first fuses the unimodal information to obtain bimodal feature information, then fuses the obtained three bimodal information to obtain the final trimodal information. The final trimodal information and each unimodal information are fused to obtain the final multimodal sentiment feature information), and each fused feature information is then passed through the TCN network layer for sequence feature extraction, and the final multimodal feature information is used for sentiment classification after information filtering and feature dimensionality reduction through the screening attention mechanism to obtain the prediction results. Experiments on the datasets CMU-MOSI (CMU multimodal opinion level sentiment intensity) and CMU-MOSEI (CMU multimodal opinion sentiment and emotion intensity) show that the model can make full use of the interaction information between different modalities, so that the accuracy of multimodal sentiment analysis can be effectively improved.

Key words: temporal convolutional networks, feature fusion, multimodal sentiment analysis, attention mechanism