Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs
AI 摘要
推理能解锁LLM的参数知识,即使对于单跳问题,推理也能提升知识回忆,但可能引入幻觉。
主要贡献
- 揭示了推理如何提升LLM的参数知识回忆能力
- 提出了计算缓冲效应和事实启动两种机制
- 指出了推理过程中的幻觉风险,并提出了缓解方法
方法论
设计了一系列假设驱动的对照实验,通过分析实验结果,探究推理对LLM知识回忆的影响及机制。
原文摘要
While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.