Banger paper from Stanford on efficient test-time scaling.

If you run agents that think for a long time, this one is worth your time.

(bookmark it)

Long reasoning keeps the entire trace in memory through full attention.

This means that the hardest problems, the ones that need the most thinking, are also the ones that cost the most to run.

The authors measured what the middle of a reasoning trace is actually worth.

Intermediate tokens steadily lose importance as the model keeps going.

Their new approach, Prefix Sliding, drops those tokens. It keeps the prefix, which holds the instructions and the available tools, plus a window of the last few thousand tokens. Everything in between gets discarded during generation.

Total memory stays capped no matter how long the model reasons.

Without any training, this runs existing models 3x faster while matching full-attention performance, and it enables RL rollouts past 100,000 tokens.

Paper: https://t.co/HzwSZ7fCdh

Chat with Paper: https://t.co/OfjtVjamIC