Google DeepMind 与多所高校发布 TheoPhysEval 基准,用 50 道由专家贡献、对世界专家约需一天的未发表理论物理题测试 AI 智能体,仅按最终数值答案评分。
基准用未发表的专家级物理题而非教材题,还比较了有无搜索的成绩,读者可据此判断模型真实研究能力和记忆程度。
New paper from Google Deep Mind, and many top Universities finds the best AI agent tested solved 49 of 50 unpublished research-level theoretical physics problems, each roughly a day's work for a world expert.
at under $3 each.
Benchmarks for AI in physics often use textbook or contest problems, or block the tools real researchers use. So it was unclear how agents handle the calculations that fill daily research.
Experts contributed 50 unpublished problems from their own research, each about a day's work for a specialist. Agents worked alone, with and without web search, and were graded only on final numerical answers.
GPT-6-Astra in Codex solved 49 of 50 with web search, and both it and Opus 5.5 in Claude Code solved 46 of 50 without it. Accuracy barely changed without search, which suggests the models have memorized much of the physics literature.
来源:Rohan Paul · x.com