Are Latent Reasoning Models Easily Interpretable?
AI 摘要
论文研究了隐式推理模型的可解释性,发现其推理token利用率低,但必要时可以解码出可理解的推理路径。
主要贡献
- 发现隐式推理模型中的推理token经常是不必要的
- 证明当推理token必要时,可以解码出 gold 推理路径
- 提出了一种无需先验知识即可从隐式token解码自然语言推理trace的方法
方法论
通过分析两个先进的隐式推理模型在逻辑推理数据集上的表现,评估其token利用率,并提出解码方法。
原文摘要
Latent reasoning models (LRMs) have attracted significant research interest due to their low inference cost (relative to explicit reasoning models) and theoretical ability to explore multiple reasoning paths in parallel. However, these benefits come at the cost of reduced interpretability: LRMs are difficult to monitor because they do not reason in natural language. This paper presents an investigation into LRM interpretability by examining two state-of-the-art LRMs. First, we find that latent reasoning tokens are often unnecessary for LRMs' predictions; on logical reasoning datasets, LRMs can almost always produce the same final answers without using latent reasoning at all. This underutilization of reasoning tokens may partially explain why LRMs do not consistently outperform explicit reasoning methods and raises doubts about the stated role of these tokens in prior work. Second, we demonstrate that when latent reasoning tokens are necessary for performance, we can decode gold reasoning traces up to 65-93% of the time for correctly predicted instances. This suggests LRMs often implement the expected solution rather than an uninterpretable reasoning process. Finally, we present a method to decode a verified natural language reasoning trace from latent tokens without knowing a gold reasoning trace a priori, demonstrating that it is possible to find a verified trace for a majority of correct predictions but only a minority of incorrect predictions. Our findings highlight that current LRMs largely encode interpretable processes, and interpretability itself can be a signal of prediction correctness.