LLM Reasoning 相关度: 9/10

QED-Nano: Teaching a Tiny Model to Prove Hard Theorems

LM-Provers, Yuxiao Qu, Amrith Setlur, Jasper Dekoninck, Edward Beeching, Jia Li, Ian Wu, Lewis Tunstall, Aviral Kumar
arXiv: 2604.04898v1 发布: 2026-04-06 更新: 2026-04-06

AI 摘要

QED-Nano通过三阶段训练,使小型模型在奥赛数学证明中达到媲美大型专有模型的性能。

主要贡献

  • 构建并开源了QED-Nano模型和训练流程
  • 提出了使用推理缓存进行长期推理的方法
  • 证明了小型模型在复杂推理任务上的竞争力

方法论

三阶段训练:有监督微调模仿证明风格;强化学习基于评分标准;推理缓存分解长证明。

原文摘要

Proprietary AI systems have recently demonstrated impressive capabilities on complex proof-based problems, with gold-level performance reported at the 2025 International Mathematical Olympiad (IMO). However, the training pipelines behind these systems remain largely undisclosed, and their reliance on large "internal" models and scaffolds makes them expensive to run, difficult to reproduce, and hard to study or improve upon. This raises a central question: can small, open models also be trained to achieve competitive reasoning performance on difficult Olympiad-level math? In this paper, we answer this question by building QED-Nano, a 4B model post-trained for Olympiad-level proofs. Our training recipe has three stages: (1) supervised fine-tuning to imbue good proof-writing styles by distilling from DeepSeek-Math-V2, (2) reinforcement learning (RL) with rubric-based rewards, and (3) expanding RL with a reasoning cache, which decomposes long proofs into iterative summarize-and-refine cycles and enables stronger test-time reasoning. QED-Nano surpasses the proof-generation performance of much larger open models, including Nomos-1 and GPT-OSS-120B, and approaches the performance of proprietary models like Gemini 3 Pro, at a fraction of the inference cost. To support further research on open mathematical reasoning, we release the full QED-Nano pipeline, including the QED-Nano and QED-Nano-SFT models, the FineProofs-SFT and FineProofs-RL datasets, and the training and evaluation code.

标签

数学推理 强化学习 模型蒸馏 开放科学

arXiv 分类

cs.AI cs.CL cs.LG