Banger paper from NVIDIA.

It's on the topic of choosing which models go into a multi-agent system.

The team compared eight selection strategies, based on size, accuracy, answer diversity and error diversity, across routing, majority vote and LLM-as-judge setups on hard science benchmarks.

Larger pools of different open models raised the theoretical best-case accuracy. Achieved accuracy often fell below the single best model in the pool.

Using several copies of one model worked better.

Majority vote over the best single model raised HLE accuracy from 29.4% to 32.2%, while nearly every mixed-model group declined.

Choosing candidates from a single model family gave the largest improvement over a standalone model of all eight strategies.

Before adding another model to a router or ensemble, measure what it adds.

Paper: https://t.co/CzF2l8AOI7

Chat with Paper: https://t.co/ozsgU5cOZn