HiVLA: A Visual-Grounded-Centric Hierarchical Embodied Manipulation System
AI 摘要
HiVLA通过解耦高层语义规划和低层运动控制,提升机器人操作的推理能力和执行精度。
主要贡献
- 提出视觉中心的分层具身操作系统HiVLA
- 引入Flow-matching Diffusion Transformer (DiT)动作专家
- 设计级联交叉注意力机制融合上下文和目标信息
方法论
使用VLM进行任务分解和视觉定位,生成结构化计划;DiT根据计划执行物理动作,通过级联交叉注意力融合信息。
原文摘要
While end-to-end Vision-Language-Action (VLA) models offer a promising paradigm for robotic manipulation, fine-tuning them on narrow control data often compromises the profound reasoning capabilities inherited from their base Vision-Language Models (VLMs). To resolve this fundamental trade-off, we propose HiVLA, a visual-grounded-centric hierarchical framework that explicitly decouples high-level semantic planning from low-level motor control. In high-level part, a VLM planner first performs task decomposition and visual grounding to generate structured plans, comprising a subtask instruction and a precise target bounding box. Then, to translate this plan into physical actions, we introduce a flow-matching Diffusion Transformer (DiT) action expert in low-level part equipped with a novel cascaded cross-attention mechanism. This design sequentially fuses global context, high-resolution object-centric crops and skill semantics, enabling the DiT to focus purely on robust execution. Our decoupled architecture preserves the VLM's zero-shot reasoning while allowing independent improvement of both components. Extensive experiments in simulation and the real world demonstrate that HiVLA significantly outperforms state-of-the-art end-to-end baselines, particularly excelling in long-horizon skill composition and the fine-grained manipulation of small objects in cluttered scenes.