Exploring Feature Interactions for Multimodal Sentiment Analysis

The study presents a sentiment analysis model to tackle two key challenges in multimodal sentiment analysis. The first challenge focuses on effectively capturing both modality-specific and modality-invariant features, which demands deep interactions across various modalities. The second challenge is to minimize interference among modalities, as such interference can degrade predictive accuracy. To address these issues, the modal feature interaction model utilizes RoBERTa and long short-term memory for feature extraction and analysis across text, audio, and video data. For the first challenge, the model employs self-attention and crossmodal attention mechanisms to facilitate modal feature interaction, enriching both intramodal and intermodal representations. To overcome the second challenge, the model reduces the L2 distance between multimodal representations during fusion, enabling seamless integration of intra- and intermodal features while capturing sentiment-related information for precise emotion prediction. Experimental results on two datasets reveal that the modal feature interaction model surpasses existing baseline models in sentiment analysis tasks.