Great paper from Anthropic on AI agents doing ML-for-science research.

(bookmark it)

If you build AI-scientist systems, this one is worth your time.

They propose Agent4Sci, which gives frontier coding agents 20 scientific ML benchmarks across 11 domains and asks them to beat the published best model.

They beat it on all 20. Claude Opus improves on the published best by 10.4% on average, and no run was flagged for misconduct.

The gains come almost entirely from standard ML-engineering tricks. The trajectories show agents misjudging what is novel, leaving compute unused, and settling into small local sweeps.

Adding three lightweight skills for research ideation, experiment management, and scheduled checklists raises Claude Opus from 10.4% to 20.1% and Codex from 4.4% to 8.1%, with more varied and more domain-relevant changes.

The models stay the same, and the research scaffolding roughly doubles the gain.

https://arxiv.org/abs/2610.28763

Chat with Paper: https://academy.dair.ai/papers/automating-ml-for-science-can-frontier-agents-climb-scientific-hills-in-the-wild-2610.28763