Multimodal Learning 相关度: 7/10

Generative World Renderer

Zheng-Hui Huang, Zhixiang Wang, Jiaming Tan, Ruihan Yu, Yidan Zhang, Bo Zheng, Yu-Lun Liu, Yung-Yu Chuang, Kaipeng Zhang
arXiv: 2604.02329v1 发布: 2026-04-02 更新: 2026-04-02

AI 摘要

提出了一个大规模高质量游戏数据集用于训练生成式渲染模型,并提出了VLM评估方法。

主要贡献

  • 大规模动态G-buffer数据集
  • VLM评估协议
  • G-buffer引导的视频生成方法

方法论

使用双屏拼接方法捕获AAA游戏中的同步RGB和G-buffer数据,并使用VLM评估渲染结果的语义一致性。

原文摘要

Scaling generative inverse and forward rendering to real-world scenarios is bottlenecked by the limited realism and temporal coherence of existing synthetic datasets. To bridge this persistent domain gap, we introduce a large-scale, dynamic dataset curated from visually complex AAA games. Using a novel dual-screen stitched capture method, we extracted 4M continuous frames (720p/30 FPS) of synchronized RGB and five G-buffer channels across diverse scenes, visual effects, and environments, including adverse weather and motion-blur variants. This dataset uniquely advances bidirectional rendering: enabling robust in-the-wild geometry and material decomposition, and facilitating high-fidelity G-buffer-guided video generation. Furthermore, to evaluate the real-world performance of inverse rendering without ground truth, we propose a novel VLM-based assessment protocol measuring semantic, spatial, and temporal consistency. Experiments demonstrate that inverse renderers fine-tuned on our data achieve superior cross-dataset generalization and controllable generation, while our VLM evaluation strongly correlates with human judgment. Combined with our toolkit, our forward renderer enables users to edit styles of AAA games from G-buffers using text prompts.

标签

生成式渲染 逆渲染 数据集 VLM G-buffer

arXiv 分类

cs.CV