Multimodal Learning 相关度: 9/10

A Contrastive Variational AutoEncoder for NSCLC Survival Prediction with Missing Modalities

Michele Zanitti, Vanja Miskovic, Francesco Trovò, Alessandra Laura Giulia Pedrocchi, Ming Shen, Yan Kyaw Tun, Arsela Prelaj, Sokol Kosta
arXiv: 2602.17402v1 发布: 2026-02-19 更新: 2026-02-19

AI 摘要

提出一种多模态对比变分自编码器,用于解决非小细胞肺癌生存预测中模态缺失问题。

主要贡献

  • 提出多模态对比变分自编码器(MCVAE)处理模态缺失问题。
  • 引入学习门控机制的融合瓶颈,标准化模态贡献。
  • 提出结合生存损失、重建损失和跨模态对比损失的多任务目标。
  • 通过随机模态掩码提高模型对任意缺失模式的鲁棒性。

方法论

使用变分自编码器捕获模态不确定性,通过对比学习对齐潜在空间,并用多任务目标和模态掩码增强模型。

原文摘要

Predicting survival outcomes for non-small cell lung cancer (NSCLC) patients is challenging due to the different individual prognostic features. This task can benefit from the integration of whole-slide images, bulk transcriptomics, and DNA methylation, which offer complementary views of the patient's condition at diagnosis. However, real-world clinical datasets are often incomplete, with entire modalities missing for a significant fraction of patients. State-of-the-art models rely on available data to create patient-level representations or use generative models to infer missing modalities, but they lack robustness in cases of severe missingness. We propose a Multimodal Contrastive Variational AutoEncoder (MCVAE) to address this issue: modality-specific variational encoders capture the uncertainty in each data source, and a fusion bottleneck with learned gating mechanisms is introduced to normalize the contributions from present modalities. We propose a multi-task objective that combines survival loss and reconstruction loss to regularize patient representations, along with a cross-modal contrastive loss that enforces cross-modal alignment in the latent space. During training, we apply stochastic modality masking to improve the robustness to arbitrary missingness patterns. Extensive evaluations on the TCGA-LUAD (n=475) and TCGA-LUSC (n=446) datasets demonstrate the efficacy of our approach in predicting disease-specific survival (DSS) and its robustness to severe missingness scenarios compared to two state-of-the-art models. Finally, we bring some clarifications on multimodal integration by testing our model on all subsets of modalities, finding that integration is not always beneficial to the task.

标签

多模态学习 生存预测 缺失数据处理 对比学习 变分自编码器

arXiv 分类

cs.AI