DuET: Dual Execution for Test Output Prediction with Generated Code and Pseudocode
AI 摘要
DuET结合代码执行和伪代码推理,提升LLM在测试输出预测任务中的准确性。
主要贡献
- 提出双执行框架DuET,结合代码执行和伪代码推理
- 利用LLM进行伪代码执行,增强预测的容错性
- 通过多数投票机制融合两种方法的优点,提高预测准确率
方法论
提出了DuET框架,使用代码直接执行和LLM推理的伪代码执行,通过多数投票确定最终输出。
原文摘要
This work addresses test output prediction, a key challenge in test case generation. To improve the reliability of predicted outputs by LLMs, prior approaches generate code first to ground predictions. One grounding strategy is direct execution of generated code, but even minor errors can cause failures. To address this, we introduce LLM-based pseudocode execution, which grounds prediction on more error-resilient pseudocode and simulates execution via LLM reasoning. We further propose DuET, a dual-execution framework that combines both approaches by functional majority voting. Our analysis shows the two approaches are complementary in overcoming the limitations of direct execution suffering from code errors, and pseudocode reasoning from hallucination. On LiveCodeBench, DuET achieves the state-of-the-art performance, improving Pass@1 by 13.6 pp.