Agent Tuning & Optimization 相关度: 7/10

RoboPocket: Improve Robot Policies Instantly with Your Phone

Junjie Fang, Wendi Chen, Han Xue, Fangyuan Zhou, Tian Le, Yi Wang, Yuting Zhang, Jun Lv, Chuan Wen, Cewu Lu
arXiv: 2603.05504v1 发布: 2026-03-05 更新: 2026-03-05

AI 摘要

RoboPocket利用手机AR进行机器人策略迭代,提升数据效率并加速在线精调。

主要贡献

  • 提出RoboPocket系统,实现无机器人策略迭代
  • 使用AR视觉预测进行远程推理,提供沉浸式反馈
  • 实现异步在线精调管道,加速策略更新

方法论

通过手机AR可视化策略预测轨迹,引导数据收集;异步在线精调持续更新策略。

原文摘要

Scaling imitation learning is fundamentally constrained by the efficiency of data collection. While handheld interfaces have emerged as a scalable solution for in-the-wild data acquisition, they predominantly operate in an open-loop manner: operators blindly collect demonstrations without knowing the underlying policy's weaknesses, leading to inefficient coverage of critical state distributions. Conversely, interactive methods like DAgger effectively address covariate shift but rely on physical robot execution, which is costly and difficult to scale. To reconcile this trade-off, we introduce RoboPocket, a portable system that enables Robot-Free Instant Policy Iteration using single consumer smartphones. Its core innovation is a Remote Inference framework that visualizes the policy's predicted trajectory via Augmented Reality (AR) Visual Foresight. This immersive feedback allows collectors to proactively identify potential failures and focus data collection on the policy's weak regions without requiring a physical robot. Furthermore, we implement an asynchronous Online Finetuning pipeline that continuously updates the policy with incoming data, effectively closing the learning loop in minutes. Extensive experiments demonstrate that RoboPocket adheres to data scaling laws and doubles the data efficiency compared to offline scaling strategies, overcoming their long-standing efficiency bottleneck. Moreover, our instant iteration loop also boosts sample efficiency by up to 2$\times$ in distributed environments a small number of interactive corrections per person. Project page and videos: https://robo-pocket.github.io.

标签

机器人学习 模仿学习 增强现实 在线学习

arXiv 分类

cs.RO cs.AI cs.LG