AI 导读
harness 工程死了吗? 绝对不是! 就像“提示词工程已死”和“RAG 已死”的说法一样,这不过是又一个为了点击量和引战而生的错误论断。 它主要源自最近发表的几篇论文,这些论文实验设置糟糕,只聚焦于已经被刷到极限的 SWE 基准上的 SWE。 如果只谈 SWE,那当然很多 harness 相关的东西正被模型吸收。这很好。这要归功于 harness 与模型协同进化的更好方法。 但现实中,你仍然需要一个稳定、最小化的 harness,才能可靠且低成本地获得高质量结果。 我看到的大问题是,我们假设 SWE 不会演进。它已经演进了,因此团队发现他们需要专门的 harness 来改进代码审查、网络安全,以及在更复杂环境(多智能体系统、沙箱等)中的工作。 harness 不只是关于提示词;它是让 AI 模型在现实世界中运作的脚手架,涵盖安全、安全护栏、授权、协调、记忆、可扩展的新基础设施、效率、成本等等。如果你在计算机科学领域待得够久,这些没有一样是固定不变的。那怎么能说脚手架/harness 不需要?更糟的是,怎么能说这些 AI 模型会知道所有这些——其中很多还是专有的、且因问题和领域而异的? 当你把智能体应用到 SWE 之外时,情况会更复杂;领域专用的 harness 正在各处涌现。这些……
正文
Is harness engineering dead?
Is harness engineering dead? Absolutely NOT! Like the "prompt engineering is dead" and "RAG is dead" claims, this is just another misinformed claim used for clicks and ragebaiting. Mostly, it stems from a few recently published papers with poor experimental setups that focus only on SWE on already benchmaxxed SWE benchmarks. If we are talking about SWE, of course a lot of the harness stuff is getting absorbed by the model. And that's great. Thanks to better approaches for co-evolving harnesses and models. But in reality, you still need a stable, minimal harness to get high-quality results reliably and affordably. The big issue I see is that we assume SWE won't evolve. It already has, and as a result, teams are finding they need specialized harnesses to improve code review, cybersecurity, and work in more complex environments (multi-agent systems, sandboxes, etc.). The harness isn't just about prompts; it's the scaffolding that makes AI models work in the real world, covering security, safety guardrails, authorization, coordination, memory, scalable and new infra, efficiency, costs, and more. If you have been in computer science for long, none of these stay fixed. So how can anyone claim the scaffold/harness isn't needed? And worse, how can anyone claim these AI models will know all that, much of it proprietary and custom to every problem and domain? It gets more complex when you apply agents beyond SWE; domain-specific harnesses are popping up everywhere. These require entirely new ways to optimize context, memory, guardrails, security, and more. To make my point even clearer, look at the new wave of proactive and personal agents (Muse, Grok Bot, dots, Instinct, etc.). They involve clever, complex harness engineering. I think we will see vertical personal agents too, so the harness will get even more specialized. I can go on and on. I'd love to hear what other serious builders here are seeing.在 X 查看被引用的帖子
来源:DAIR.AI · x.com