Multimodal Learning 相关度: 9/10

QA-MoE: Towards a Continuous Reliability Spectrum with Quality-Aware Mixture of Experts for Robust Multimodal Sentiment Analysis

Yitong Zhu, Yuxuan Jiang, Guanxuan Jiang, Bojing Hou, Peng Yuan Zhou, Ge Lin Kan, Yuyang Wang
arXiv: 2604.05704v1 发布: 2026-04-07 更新: 2026-04-07

AI 摘要

提出QA-MoE模型,利用质量感知的专家混合方法,提升多模态情感分析在噪声环境下的鲁棒性。

主要贡献

  • 提出了连续可靠性谱,统一缺失和质量退化
  • 提出了QA-MoE框架,通过不确定性引导专家路由
  • 实现了在多种退化场景下的state-of-the-art性能

方法论

利用自监督学习量化模态可靠性,通过专家混合机制抑制不可靠信号的误差传播,保留相关信息。

原文摘要

Multimodal Sentiment Analysis (MSA) aims to infer human sentiment from textual, acoustic, and visual signals. In real-world scenarios, however, multimodal inputs are often compromised by dynamic noise or modality missingness. Existing methods typically treat these imperfections as discrete cases or assume fixed corruption ratios, which limits their adaptability to continuously varying reliability conditions. To address this, we first introduce a Continuous Reliability Spectrum to unify missingness and quality degradation into a single framework. Building on this, we propose QA-MoE, a Quality-Aware Mixture-of-Experts framework that quantifies modality reliability via self-supervised aleatoric uncertainty. This mechanism explicitly guides expert routing, enabling the model to suppress error propagation from unreliable signals while preserving task-relevant information. Extensive experiments indicate that QA-MoE achieves competitive or state-of-the-art performance across diverse degradation scenarios and exhibits a promising One-Checkpoint-for-All property in practice.

标签

多模态情感分析 专家混合 不确定性量化 鲁棒性

arXiv 分类

cs.AI