Multimodal Learning 相关度: 9/10

GeoAlignCLIP: Enhancing Fine-Grained Vision-Language Alignment in Remote Sensing via Multi-Granular Consistency Learning

Xiao Yang, Ronghao Fu, Zhuoran Duan, Zhiwen Lin, Xueyan Liu, Bo Yang
arXiv: 2603.09566v1 发布: 2026-03-10 更新: 2026-03-10

AI 摘要

GeoAlignCLIP通过多粒度一致性学习增强遥感图像中文本对齐,提升细粒度视觉语言理解。

主要贡献

  • 提出了GeoAlignCLIP框架,实现遥感图像中细粒度对齐
  • 学习多粒度语义对齐并结合模内一致性
  • 构建了RSFG-100k细粒度遥感数据集

方法论

通过多粒度语义对齐和模内一致性学习,GeoAlignCLIP实现了图像区域和文本概念之间更精确的视觉语义对齐。

原文摘要

Vision-language pretraining models have made significant progress in bridging remote sensing imagery with natural language. However, existing approaches often fail to effectively integrate multi-granular visual and textual information, relying primarily on global image-text alignment. This limitation hinders the model's ability to accurately capture fine-grained details in images, thus restricting its performance in complex, fine-grained tasks. To address this, we propose GeoAlignCLIP, a unified framework that achieves fine-grained alignment in remote sensing tasks by learning multi-granular semantic alignments and incorporating intra-modal consistency, enabling more precise visual-semantic alignment between image regions and text concepts. Additionally, we construct RSFG-100k, a fine-granular remote sensing dataset containing scene descriptions, region-level annotations, and challenging hard-negative samples, providing hierarchical supervision for model training. Extensive experiments conducted on multiple public remote-sensing benchmarks demonstrate that GeoAlignCLIP consistently outperforms existing RS-specific methods across diverse tasks, exhibiting more robust and accurate fine-grained vision-language alignment.

标签

遥感 视觉语言 多模态 细粒度

arXiv 分类

cs.CV