跳到正文
🚨 AI News | TestingCatalog· @testingcatalog · X·· 3 小时前AI 评分44
AI 导读

Anthropic 推出 self-assessment 技术,让 Claude 在回复后立即对对话做初步安全评估。在 Claude Fable 5 上,该机制以 1% 误报率识别出强提示词分类器标记为潜在有害对话的 78%,高优先级对话达 94%。Anthropic 称每次成功攻击约消耗无防护时 4 倍的生成 token。

正文

Anthropic introduced self-assessment, a new technique that lets Claude give each conversation a preliminary safety assessment right after it responds.

> On Claude Fable 5, self-assessment flags 78% of the conversations that a stronger, prompted classifier flags as potentially harmful, and 94% of the high-priority ones, at a 1% false-positive rate.

Anthropic also shed light on how self-assessment and prompted classifiers help block attacks.

> "Each successful attack costs roughly four times as many generated tokens as with no safeguards."

We already live in a world where your security depends on the compute allocated to defend the systems you rely on.

引用Anthropic@AnthropicAI
We’ve trained Claude to flag potential misuse in conversations, a technique we call self-assessment. Right after it responds, Claude gives a preliminary safety assessment of the conversation. Since we began using self-assessment in June, it has surfaced misuse that our other safeguards weren’t designed to catch. Because Claude has already read the conversation, self-assessment is very efficient. And unlike classifiers, which target a discrete set of harms, it draws on Claude’s own judgment, so it gets more accurate as our models improve. Read more on our new Safeguards blog: https://safeguards.anthropic.com/2026/self-assessment/
在 X 查看被引用的帖子

来源:🚨 AI News | TestingCatalog · x.com