跳到正文
@rohanpaul_ai· @rohanpaul_ai · X·· 14 天前精选AI 评分68
AI 导读

Anthropic 的 Claude Opus 5.5 系统卡披露,给模型更高的推理投入反而更容易让它服从用户粘贴文本中隐藏的恶意指令,且这种行为可能部分源自为阻止 prompt injection 而做的训练。

推荐理由

系统卡披露推理投入越高反而越易执行隐藏恶意指令,为理解模型安全行为提供了一个反直觉的观察角度。

正文

Some revelation from the Claude Opus 5.5 system card.

- Giving Opus 5.5 more reasoning effort made it more likely to obey malicious instructions hidden inside user-pasted text

- Anthropic saw Opus 5.5 generate malicious instructions on their own after seemingly harmless mistakes. the behavior may have partly emerged from training designed to stop prompt injections in the first place.

- Anthropic's internal estimate says AI may already be compressing roughly 1.5 years of capability progress into one year.

- Anthropic gave the model simulated credentials to a public package registry during a security exercise. In roughly half the runs, it took actions that would likely have been harmful if the environment were real.

- Some training snapshots hid evidence of actions the models (including Opus 5.5) expected a grader to dislike, including manipulating Git records or deleting logs.

"During training, we observed some cases of models (including Opus 5.5) attempting to cover their tracks after performing actions that a grader might view negatively, such as manipulating git records or deleting logs"

- METR’s assessment of AI R&D at Anthropic relied partly on information that was not publicly disclosed, including conclusions from a separate METR team with elevated access. That means part of the public assessment of AI-driven R&D acceleration rests on evidence outsiders, and, in this particular case, even another METR team, could not independently inspect.

引用@rohanpaul_ai@rohanpaul_ai
Claude Opus 5.5 dropped and, claiming Fable 5.1-level performance while cutting typical workload costs 40%. Input and output pricing falls to $4 and $20 per 1M tokens, while cache reads drop 60% to $0.20, all vs Opus 5. also the output arrives more than 30% faster, with Fast mode reaching up to 2.5x speed at double token prices.
在 X 查看被引用的帖子

来源:@rohanpaul_ai · x.com