Super interesting finding on optimizing agent harnesses.

When you optimize an agent harness for both quality and cost, apply your cost preference only at the end of the search, since applying it at every step throws away harnesses that lead to better ones.

That is the key insight behind Microsoft's ParetoSaddler.

At each step, an optimizer edits a parent harness. Both versions run several times, and a Bayesian bootstrap over those runs decides whether the candidate shows credible progress in quality or in cost. Promoted candidates stay available as parents, and the final deployment choice is made separately.

On GAIA2, quality goes from 56.22% to 62.89% while cost per task drops from $0.78 to $0.45. On ClawMark, quality goes from 70.86% to 81.35%, and cost drops from $1.17 to $1.01.

The Pareto-selected harnesses also beat quality-only optimization on both quality and cost.

If your harness search only tracks accuracy, add repeated runs and a cost axis before deciding which candidates to keep.

https://arxiv.org/abs/2611.02079

Chat with Paper: https://academy.dair.ai/papers/paretosaddler-pareto-guided-optimization-of-agent-harnesses-for-quality-and-cost-2611.02079