What Really Controls Temporal Reasoning in Large Language Models: Tokenisation or Representation of Time?
AI 摘要
该论文提出了一个多语言时间推理基准,分析了token化和时间表示对LLM时间推理的影响。
主要贡献
- 提出了MultiTempBench多语言时间推理基准
- 发现token化质量是低资源语言时间推理的瓶颈
- 分析了时间线性度与时间推理性能的关系
方法论
构建包含多种语言和日历格式的时间推理数据集,评估20个LLM,并提出多语言Date Fragmentation Ratio指标。
原文摘要
We present MultiTempBench, a multilingual temporal reasoning benchmark spanning three tasks, date arithmetic, time zone conversion, and temporal relation extraction across five languages (English, German, Chinese, Arabic, and Hausa) and multiple calendar conventions (Gregorian, Hijri, and Chinese Lunar). MultiTempBench contains $15,000$ examples built by translating $750$ curated English questions and expanding each into controlled date-format variants. We evaluate 20 LLMs and introduce the multilingual Date Fragmentation Ratio (mDFR), calibrated with human severity ratings, together with geometric-probing analyses of internal temporal representations. We find tokenisation quality of temporal artefacts is a resource-dependent bottleneck: in low-resource languages and rarer calendar formats, fragmentation disrupts Year/Month/Day separation and accuracy collapses, while high-resource settings are often robust to digit-level splitting. Beyond tokenisation, crossed mixed-effects regression shows that temporal linearity is the strongest predictor of temporal reasoning in high-resource languages, whereas fragmentation is the stronger predictor in low-resource languages. Code is available at: https://github.com/gagan3012/mtb