A model can look strong on average and still be unpredictable on the same prompt.

Martian’s new AI Frontier measures that gap with 10 runs per datapoint across 16 benchmarks. Its current reliability table puts Qwen3.7 Max at 96.1%, Claude Opus 4.6 at 94.4%, and GPT-5.5 at 93.5%.

Reliability here means consistency in being right or wrong, not raw accuracy. In an agent workflow, one unstable step can derail everything that follows.

The paper behind the dashboard found a 54% error reduction at matched cost using oracle routing across models versus each benchmark’s best single model.