arXiv 论文（arxiv.org/abs/2610.40488）让六个前沿模型评估七篇近期 NLP 论文并运行其提出的全部实验。415 项实验中 194 项被认为可靠、有价值且新颖，仅 73 项超出种子论文且非显而易见。模型的首选优于随机挑选，但未证明优于排名第一的实验；预测优于基础比率，却不比只读论文的同一模型新实例更好；多智能体团队成本更高但提案质量无明显提升。
Regarding autoresearch agents, this question has attracted a lot of attention.
Do research agents have taste?
This has attracted massive interest because autoresearch agents could unlock major discoveries.
This study gave six frontier models seven recent NLP papers, then ran every experiment they proposed, including the ones they ranked last.
Of 415 experiments, 194 were judged sound, valuable, and new. Only 73 were also beyond the seed paper and non-obvious.
The rankings added little. First choices beat a random pick, but they were not shown to beat the experiment written first.
Forecasts beat a base rate, but they were not measurably better than a fresh instance of the same model that read only the paper and registrations.
They tested with teams of agents, which cost more, with no clear improvement in proposal quality.
https://arxiv.org/abs/2610.40488
Chat with Paper: https://academy.dair.ai/papers/rtaste-do-research-agents-have-taste-evaluating-early-stage-research-judgment-2610.40488
来源：elvis · x.com