Nice paper to improve inference efficiency.

It's been a while we haven't seen good work on efficiency.

Here is why it matters:

A long-running agent's workspace outgrows its context window long before the task finishes.

The first approach commonly used, compaction, loses the fine-grained execution evidence. And text retrieval re-prefills content the model already processed.

KVMem keeps the overflow as paged KV state instead, spread across GPU memory, host memory and NVMe.

Lightweight attention-space indexes, native to the model, pick the relevant historical blocks and materialize a query-dependent view that fits inside the native context window.

On the DeepSWE long-context test with Qwen3.8-27B, task success goes from 43.8% under compaction to 48.4%.

The local deployment result stands out. It runs Qwen3.6/3.8-27B NVFP4 with MTP on a laptop with a 24GB RTX 5090, virtualizing an agent workspace up to 1M tokens, four times the model's native 256K window, at around 50 tokens per second.

Paper: https://t.co/rKIAqnzxII