This paper shows a different way to make chain-of-thought cheaper: move reusable reasoning from generation into the prompt.

Instead of asking an LLM to regenerate every reasoning step, give it the relevant reasoning pattern upfront and let it think shorter.

Memory-Augmented Compression is training-free: it distills solved examples into reusable reasoning memories, retrieves relevant ones for each query, and injects them before compressed reasoning.

That shifts some work from slow autoregressive decoding to the more parallel prefill stage.

With Qwen2.5-7B, adding memory to Chain-of-Draft recovered 21.4 accuracy points on GSM8K and 28.0 on MATH, while model latency stayed 1.49× and 1.14× faster than standard CoT.