Multimodal Learning 相关度: 9/10

Bridging MRI and PET physiology: Untangling complementarity through orthogonal representations

Sonja Adomeit, Kartikay Tehlan, Lukas Förner, Katharina Weisser, Helen Scholtiseek, David Kaufmann, Julie Steinestel, Constantin Lapa, Thomas Kröncke, Thomas Wendler
arXiv: 2604.07154v1 发布: 2026-04-08 更新: 2026-04-08

AI 摘要

提出一种基于正交子空间分解的多模态融合方法,区分MRI和PET的互补信息。

主要贡献

  • 提出正交子空间分解框架用于多模态融合
  • 使用隐式神经表示(INR)建模MRI与PET的映射关系
  • 引入基于SVD的正则化项强制正交性

方法论

使用INR学习MRI到PET的映射,并通过SVD正则化强制MRI特征空间与PET残差的正交性。

原文摘要

Multimodal imaging analysis often relies on joint latent representations, yet these approaches rarely define what information is shared versus modality-specific. Clarifying this distinction is clinically relevant, as it delineates the irreducible contribution of each modality and informs rational acquisition strategies. We propose a subspace decomposition framework that reframes multimodal fusion as a problem of orthogonal subspace separation rather than translation. We decompose Prostate-Specific Membrane Antigen (PSMA) PET uptake into an MRI-explainable physiological envelope and an orthogonal residual reflecting signal components not expressible within the MRI feature manifold. Using multiparametric MRI, we train an intensity-based, non-spatial implicit neural representation (INR) to map MRI feature vectors to PET uptake. We introduce a projection-based regularization using singular value decomposition to penalize residual components lying within the span of the MRI feature manifold. This enforces mathematical orthogonality between tissue-level physiological properties (structure, diffusion, perfusion) and intracellular PSMA expression. Tested on 13 prostate cancer patients, the model demonstrates that residual components spanned by MRI features are absorbed into the learned envelope, while the orthogonal residual is largest in tumour regions. This indicates that PSMA PET contains signal components not recoverable from MRI-derived physiological descriptors. The resulting decomposition provides a structured characterization of modality complementarity grounded in representation geometry rather than image translation.

标签

多模态学习 图像融合 子空间分解 隐式神经表示

arXiv 分类

cs.CV cs.AI