HGP-Mamba: Integrating Histology and Generated Protein Features for Mamba-based Multimodal Survival Risk Prediction
AI 摘要
HGP-Mamba是一种基于Mamba的多模态框架,结合组织学和生成的蛋白特征进行癌症生存风险预测。
主要贡献
- 提出一种高效的蛋白特征提取器 (PFE)
- 引入Local Interaction-aware Mamba (LiAM) 用于细粒度特征交互
- 引入Global Interaction-enhanced Mamba (GiEM) 用于促进全局模态融合
方法论
使用预训练模型从组织切片图像生成蛋白嵌入,结合组织学嵌入,通过LiAM和GiEM进行多模态融合和生存风险预测。
原文摘要
Recent advances in multimodal learning have significantly improved cancer survival risk prediction. However, the joint prognostic potential of protein markers and histopathology images remains underexplored, largely due to the high cost and limited availability of protein expression profiling. To address this challenge, we propose HGP-Mamba, a Mamba-based multimodal framework that efficiently integrates histological with generated protein features for survival risk prediction. Specifically, we introduce a protein feature extractor (PFE) that leverages pretrained foundation models to derive high-throughput protein embeddings directly from Whole Slide Images (WSIs), enabling data-efficient incorporation of molecular information. Together with histology embeddings that capture morphological patterns, we further introduce the Local Interaction-aware Mamba (LiAM) for fine-grained feature interaction and the Global Interaction-enhanced Mamba (GiEM) to promote holistic modality fusion at the slide level, thus capture complex cross-modal dependencies. Experiments on four public cancer datasets demonstrate that HGP-Mamba achieves state-of-the-art performance while maintaining superior computational efficiency compared with existing methods. Our source code is publicly available at <a href="https://github.com/Daijing-ai/HGP-Mamba.git">this https URL</a>.