Multimodal Learning 相关度: 9/10

IntroSVG: Learning from Rendering Feedback for Text-to-SVG Generation via an Introspective Generator-Critic Framework

Feiyu Wang, Jiayuan Yang, Zhiyuan Zhao, Da Zhang, Bingyu Li, Peng Liu, Junyu Gao
arXiv: 2603.09312v1 发布: 2026-03-10 更新: 2026-03-10

AI 摘要

IntroSVG通过生成器-评论家框架,结合渲染反馈,提升文本到SVG的生成质量。

主要贡献

  • 提出Introspective SVG Generation Framework (IntroSVG)
  • 使用统一的VLM作为生成器和评论家
  • 利用早期失败数据提升模型鲁棒性

方法论

采用VLM的生成器-评论家框架,通过SFT学习SVG生成和反馈,再用DPO对齐生成器策略,迭代优化。

原文摘要

Scalable Vector Graphics (SVG) are central to digital design due to their inherent scalability and editability. Despite significant advancements in content generation enabled by Visual Language Models (VLMs), existing text-to-SVG generation methods are limited by a core challenge: the autoregressive training process does not incorporate visual perception of the final rendered image, which fundamentally constrains generation quality. To address this limitation, we propose an Introspective SVG Generation Framework (IntroSVG). At its core, the framework instantiates a unified VLM that operates in a closed loop, assuming dual roles of both generator and critic. Specifically, through Supervised Fine-Tuning (SFT), the model learns to draft SVGs and to provide feedback on their rendered outputs; moreover, we systematically convert early-stage failures into high-quality error-correction training data, thereby enhancing model robustness. Subsequently, we leverage a high-capacity teacher VLM to construct a preference dataset and further align the generator's policy through Direct Preference Optimization (DPO). During inference, the optimized generator and critic operate collaboratively in an iterative "generate-review-refine" cycle, starting from imperfect intermediate drafts to autonomously improve output quality. Experimental results demonstrate that our method achieves state-of-the-art performance across several key evaluation metrics, generating SVGs with more complex structures, stronger semantic alignment, and greater editability. These results corroborate the effectiveness of incorporating explicit visual feedback into the generation loop.

标签

SVG生成 VLM 生成器-评论家 渲染反馈 自监督学习

arXiv 分类

cs.CV