LLM Reasoning 相关度: 9/10

Why Does Self-Distillation (Sometimes) Degrade the Reasoning Capability of LLMs?

Jeonghye Kim, Xufang Luo, Minbeom Kim, Sangmook Lee, Dohyung Kim, Jiwon Jeon, Dongsheng Li, Yuqing Yang
arXiv: 2603.24472v1 发布: 2026-03-25 更新: 2026-03-25

AI 摘要

自蒸馏有时会降低LLM的推理能力,主要是因为抑制了模型在推理过程中的不确定性表达。

主要贡献

  • 发现自蒸馏会降低LLM的推理能力
  • 指出原因是抑制了模型的不确定性表达
  • 通过实验验证了上下文信息丰富度和任务覆盖范围对推理的影响

方法论

通过控制上下文信息丰富度和任务覆盖范围,进行对比实验,分析自蒸馏对LLM推理能力的影响。

原文摘要

Self-distillation has emerged as an effective post-training paradigm for LLMs, often improving performance while shortening reasoning traces. However, in mathematical reasoning, we find that it can reduce response length while degrading performance. We trace this degradation to the suppression of epistemic verbalization - the model's expression of uncertainty during reasoning. Through controlled experiments varying conditioning context richness and task coverage, we show that conditioning the teacher on rich information suppresses uncertainty expression, enabling rapid in-domain optimization with limited task coverage but harming OOD performance, where unseen problems benefit from expressing uncertainty and adjusting accordingly. Across Qwen3-8B, DeepSeek-Distill-Qwen-7B, and Olmo3-7B-Instruct, we observe performance drops of up to 40%. Our findings highlight that exposing appropriate levels of uncertainty is crucial for robust reasoning and underscore the importance of optimizing reasoning behavior beyond merely reinforcing correct answer traces.

标签

自蒸馏 LLM 推理 不确定性 数学推理

arXiv 分类

cs.CL cs.LG