Multimodal Learning 相关度: 8/10

Trimodal Deep Learning for Glioma Survival Prediction: A Feasibility Study Integrating Histopathology, Gene Expression, and MRI

Iain Swift, JingHua Ye
arXiv: 2603.29968v1 发布: 2026-03-31 更新: 2026-03-31

AI 摘要

研究使用三模态深度学习(病理、基因、MRI)预测脑胶质瘤患者生存期,初步验证了MRI的潜在价值。

主要贡献

  • 探索MRI在脑胶质瘤生存预测中的作用
  • 提出融合病理、基因表达和MRI的三模态深度学习框架
  • 初步验证了三模态融合的可行性和潜在提升

方法论

采用深度学习,融合组织病理学、基因表达和MRI数据,比较单模态、双模态和三模态模型的生存预测效果。

原文摘要

Multimodal deep learning has improved prognostic accuracy for brain tumours by integrating histopathology and genomic data, yet the contribution of volumetric MRI within unified survival frameworks remains unexplored. This pilot study extends a bimodal framework by incorporating Fluid Attenuated Inversion Recovery (FLAIR) MRI from BraTS2021 as a third modality. Using the TCGA-GBMLGG cohort (664 patients), we evaluate three unimodal models, nine bimodal configurations, and three trimodal configurations across early, late, and joint fusion strategies. In this small cohort setting, trimodal early fusion achieves an exploratory Composite Score (CS = 0.854), with a controlled $Δ$CS of +0.011 over the bimodal baseline on identical patients, though this difference is not statistically significant (p = 0.250, permutation test). MRI achieves reasonable unimodal discrimination (CS = 0.755) but does not substantially improve bimodal pairs, while providing measurable uplift in the three-way combination. All MRI containing experiments are constrained to 19 test patients, yielding wide bootstrap confidence intervals (e.g. [0.400,1.000]) that preclude definitive conclusions. These findings provide preliminary evidence that a third imaging modality may add prognostic value even with limited sample sizes, and that additional modalities require sufficient multimodal context to contribute effectively.

标签

多模态学习 深度学习 脑胶质瘤 生存预测 MRI

arXiv 分类

cs.CV cs.AI