Multimodal Learning 相关度: 9/10

A ROS 2 Wrapper for Florence-2: Multi-Mode Local Vision-Language Inference for Robotic Systems

J. E. Domínguez-Vidal
arXiv: 2604.01179v1 发布: 2026-04-01 更新: 2026-04-01

AI 摘要

开发了Florence-2模型的ROS 2封装,支持多种交互模式,方便机器人系统集成视觉-语言模型。

主要贡献

  • 提供 Florence-2 模型的 ROS 2 封装
  • 支持连续、同步和异步三种交互模式
  • 提供本地部署和Docker部署方式

方法论

通过ROS 2接口封装Florence-2模型,并针对不同交互模式设计了不同的ROS 2节点和服务。

原文摘要

Foundation vision-language models are becoming increasingly relevant to robotics because they can provide richer semantic perception than narrow task-specific pipelines. However, their practical adoption in robot software stacks still depends on reproducible middleware integrations rather than on model quality alone. Florence-2 is especially attractive in this regard because it unifies captioning, optical character recognition, open-vocabulary detection, grounding and related vision-language tasks within a comparatively manageable model size. This article presents a ROS 2 wrapper for Florence-2 that exposes the model through three complementary interaction modes: continuous topic-driven processing, synchronous service calls and asynchronous actions. The wrapper is designed for local execution and supports both native installation and Docker container deployment. It also combines generic JSON outputs with standard ROS 2 message bindings for detection-oriented tasks. A functional validation is reported together with a throughput study on several GPUs, showing that local deployment is feasible with consumer grade hardware. The repository is publicly available here: https://github.com/JEDominguezVidal/florence2_ros2_wrapper

标签

ROS 2 Florence-2 Vision-Language Model

arXiv 分类

cs.RO cs.AI cs.CV