Interesting paper from Microsoft on context-guided model routing.

They found a nice trick to achieve cost-aware model routing for agents in a multi-turn setting at the user-turn level.

If you route agent turns between a cheap model and an expensive one, they suggest giving the router the session history.

Microsoft tested zero-shot routing on multi-turn spreadsheet agents.

At the same offloading rate, a router that reads the history passes 3.3 points more tasks than one that sees only the current request.

Prompt caching adds a cost to every switch. Switching from the expensive model to the cheap one recovers its cache cost within two calls. Switching up did not break even in their tests.

The cheap model's cost advantage also shrank from 6.8x to 3.2x on longer sessions.

https://arxiv.org/abs/2610.19292

Chat with Paper: https://academy.dair.ai/papers/whose-turn-is-it-context-guided-model-routing-for-stateful-agents-2610.19292