Multimodal Learning 相关度: 9/10

HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction

Jing Dai, Chen Wu, Ming Wu, Qibin Zhang, Zexi Wu, Jingdong Zhang, Hongming Xu
arXiv: 2603.16421v1 发布: 2026-03-17 更新: 2026-03-17

AI 摘要

HGP-Mamba是一种基于Mamba的多模态框架,结合组织学和生成的蛋白特征进行癌症生存风险预测。

主要贡献

  • 提出一种高效的蛋白特征提取器 (PFE)
  • 引入Local Interaction-aware Mamba (LiAM) 用于细粒度特征交互
  • 引入Global Interaction-enhanced Mamba (GiEM) 用于促进全局模态融合

方法论

使用预训练模型从组织切片图像生成蛋白嵌入,结合组织学嵌入,通过LiAM和GiEM进行多模态融合和生存风险预测。

原文摘要

Recent advances in multimodal learning have significantly improved cancer survival risk prediction. However, the joint prognostic potential of protein markers and histopathology images remains underexplored, largely due to the high cost and limited availability of protein expression profiling. To address this challenge, we propose HGP-Mamba, a Mamba-based multimodal framework that efficiently integrates histological with generated protein features for survival risk prediction. Specifically, we introduce a protein feature extractor (PFE) that leverages pretrained foundation models to derive high-throughput protein embeddings directly from Whole Slide Images (WSIs), enabling data-efficient incorporation of molecular information. Together with histology embeddings that capture morphological patterns, we further introduce the Local Interaction-aware Mamba (LiAM) for fine-grained feature interaction and the Global Interaction-enhanced Mamba (GiEM) to promote holistic modality fusion at the slide level, thus capture complex cross-modal dependencies. Experiments on four public cancer datasets demonstrate that HGP-Mamba achieves state-of-the-art performance while maintaining superior computational efficiency compared with existing methods. Our source code is publicly available at <a href="https://github.com/Daijing-ai/HGP-Mamba.git">this https URL</a>.

标签

多模态学习 生存风险预测 Mamba 组织病理学 蛋白质组学

arXiv 分类

cs.CV