论文《Trace as State》提出让长上下文 Transformer 带着自己此前的推理轨迹重读一遍输入,在 27 项对比中赢了 26 项。原因是模型按顺序处理提示词,往往太晚才意识到关键信息,无法回头改变对前文的理解;把推理轨迹前置后,模型重读时就知道该找什么。
For hard long-context tasks, the paper suggests a simple fix: run the model again with its previous reasoning first, an approach that won 26 of 27 comparisons.
Because, Long-context models often figure out what matters too late
Because it processes the prompt in order, that late insight cannot go back and change how the earlier context was read.
TRACE AS STATE gives it another pass, with its own earlier reasoning placed first, so it knows what to look for while rereading.
– arxiv. org/abs/2609.02702
Title: "Trace as State: Reasoning Traces as Conditional States for Long-Context Transformers"
来源:@rohanpaul_ai · x.com