Publication

Abstract : The rapid growth of e-commerce has transformed customer feedback into a multimodal medium, encompassing not only text but also speech, images, and video reviews. Traditional unimodal sentiment analysis approaches—predominantly text-based,—fail to capture the rich emotional cues embedded in voice tone, facial expressions, and visual context, often resulting in misclassifications in cases of irony or ambiguous expressions. Recent multimodal sentiment analysis (MSA) studies have explored fusion strategies across text, audio, and video, however most existing methods either rely on computationally heavy architectures or prioritize text while underutilizing non-textual modalities. This study introduced the Audio–Video Squeeze-Excitation Network (AViSE-Net), a lightweight yet discriminative multimodal framework that leverages dilated CNNs for audio and an SE-enhanced MobileNetV2 backbone for video and image data. Key innovations include 1) dilated convolutions in the audio branch for efficient long-range temporal modelling, 2) channel-wise recalibration through squeeze-and-excitation blocks in the video pathway, 3) fusion with LayerNorm and GELU activations for stable cross-modal integration, and 4) a progressive unfreezing strategy to accelerate convergence while preserving pretrained visual features. Evaluated on a curated multimodal dataset of customer product reviews, AViSE-Net achieves 97.9% accuracy with ~50% fewer parameters than heavy baseline models such as BiLSTM+ ResNet18, while maintaining real-time inference speeds of ~42 fps on a GPU. Comparative experiments and ablation studies confirmed that each architectural enhancement contributes significantly to superior performance. These results establish AViSE-Net as a compact, scalable, and deployment-ready solution for real-world multimodal sentiment analysis in e-commerce and consumer feedback applications.