Multimodal Learning 相关度: 9/10

CoCo: Code as CoT for Text-to-Image Preview and Rare Concept Generation

Haodong Li, Chunmei Qing, Huanyu Zhang, Dongzhi Jiang, Yihang Zou, Hongbo Peng, Dingming Li, Yuhong Dai, ZePeng Lin, Juanxi Tian, Yi Zhou, Siqi Dai, Jingwei Wu
arXiv: 2603.08652v1 发布: 2026-03-09 更新: 2026-03-09

AI 摘要

CoCo提出一种代码驱动的CoT推理框架,用于精确、可控的文本到图像生成,并构建了CoCo-10K数据集。

主要贡献

  • 提出Code-as-CoT (CoCo) 框架
  • 构建了CoCo-10K数据集
  • 在多个benchmark上验证了CoCo的有效性

方法论

CoCo首先生成可执行代码来规划场景布局,执行代码渲染草图,然后通过图像编辑进行细化,最终生成高质量图像。

原文摘要

Recent advancements in Unified Multimodal Models (UMMs) have significantly advanced text-to-image (T2I) generation, particularly through the integration of Chain-of-Thought (CoT) reasoning. However, existing CoT-based T2I methods largely rely on abstract natural-language planning, which lacks the precision required for complex spatial layouts, structured visual elements, and dense textual content. In this work, we propose CoCo (Code-as-CoT), a code-driven reasoning framework that represents the reasoning process as executable code, enabling explicit and verifiable intermediate planning for image generation. Given a text prompt, CoCo first generates executable code that specifies the structural layout of the scene, which is then executed in a sandboxed environment to render a deterministic draft image. The model subsequently refines this draft through fine-grained image editing to produce the final high-fidelity result. To support this training paradigm, we construct CoCo-10K, a curated dataset containing structured draft-final image pairs designed to teach both structured draft construction and corrective visual refinement. Empirical evaluations on StructT2IBench, OneIG-Bench, and LongText-Bench show that CoCo achieves improvements of +68.83%, +54.8%, and +41.23% over direct generation, while also outperforming other generation methods empowered by CoT. These results demonstrate that executable code is an effective and reliable reasoning paradigm for precise, controllable, and structured text-to-image generation. The code is available at: https://github.com/micky-li-hd/CoCo

标签

Text-to-Image Chain-of-Thought Code Generation Multimodal Learning Image Editing

arXiv 分类

cs.AI