Please wait ...
0% Complete
فارسی
Home
/
شانزدهمین کنفرانس بین المللی فناوری اطلاعات و دانش
Enhancing Persian Speech Emotion Recognition with Contrastive Learning and Multimodal Fusion
Authors :
Mobina Esmaeili
1
Vajiheh Sabeti
2
1- دانشگاه الزهرا(س)
2- دانشگاه الزهرا(س)
Keywords :
Multimodal Emotion Recognitiont،Representation Learning،Representation Learning،Speech-Text Fusion،ShEMO Dataset
Abstract :
Emotion recognition from both speech and text in low-resource languages such as Persian presents significant challenges due to linguistic complexity and the scarcity of labeled datasets. Conventional multimodal fusion methods often struggle to capture nuanced cross-modal interactions and typically neglect inter-class emotional relationships. To address these limitations, this paper introduces a novel contrastive learning framework that employs pre-trained projection networks to enhance multimodal representations through a combination of intra-modal, inter-modal, and semi-contrastive objectives. The refined embeddings are integrated via a lightweight fusion layer for final emotion classification. In addition, an automatic speech recognition (ASR) system is incorporated to enrich textual inputs and improve linguistic diversity. Experiments on the ShEMO corpus demonstrate that the proposed approach achieves an accuracy of 83.04% and an unweighted average recall (UAR) of 88.1%, substantially outperforming traditional fusion-based baselines. These results confirm the effectiveness of the framework in improving cross-modal alignment and representation quality, highlighting its potential for intelligent interactive systems, social media sentiment analysis, and automated affective computing applications.
Papers List
List of archived papers
ارائه مدل یادگیری ماشین برای پیشبینی سریزمانی باینری از دیدگاه مسئلههای دستهبندی با کاربرد در پیشبینی نتهای موسیقی
نیلوفر ع��دلخانی - حسام عمرانپور
بهبود کارایی بارسپاری در شبکه های سلولی با استفاده از ارتباطات مشارکتی در لایه MAC
نبیل الراشدی - رسول صادقی - وائل حسین اللامی - مهدی حمیدخانی
Context Awareness Gate for Retrieval Augmented Generation
Mohammad Hassan Heydari - Arshia Hemmat - Erfan Naman - Afsaneh Fatemi
Business Process Improvement Challenges: A Systematic Literature Review
Hanieh Kashfi - Fereidoon Shams Aliee
A perceptual loss for screen content image super-resolution
Hossein Sekhavaty-Moghadam - Marzieh Hosseinkhani - Dr Azadeh Mansouri
ارائه یک مدل تصمیم گیری چند معیاره فازی به منظور بهبود دقت فرایند تصمیم گیری به هنگام اختلال هوانوردی
فاطمه عطا عبدالرزاق - نگار مجمع
A Comparison between Slimed Network and Pruned Network for Head Pose Estimation
Amir Salimiparsa - Hadi Veisi - Mohammad-shahram Moin
An Optimized GBDT-Based Model Using SMOTE for Effective Diagnosis of Coronary Heart Disease
Elahe Moradi - Mohammad Javadian
Automatic Analysis of Inconsistencies in Inter-Enterprise Business Processes: Introducing a Formal Adaptation Patterns Catalog
Somayeh Ashourian - Shohreh َAjoudanian
A Fuzzy Cluster-Based Routing Algorithm to Extend Wireless Sensor Network Lifetime
Mostafa Mirzaie - Armin Mazinani - Dr Sayyed Majid Mazinani
more
Samin Hamayesh - Version 45.0.1