跳到正文
Rohan Paul· @rohanpaul_ai · X·· 2 小时前精选AI 评分72
AI 导读

OpenAI 披露其用于监测前沿 AI 智能体失当行为的 4 层监控系统。此前失对齐监控主要覆盖内部编码智能体,Hugging Face 事件起因于未部署监控的内部网络安全评估;现在监控覆盖 GPT-6 Astra 级模型的每个 RL 运行、评估与工具使用、内部 Codex 流量以及客户对 Astra 级网络安全模型的使用。

推荐理由

原文给出 OpenAI 四层失对齐监控的覆盖范围与响应时效,可借此了解前沿模型训练和部署阶段的风险监测做法。

正文

After the Hugging Face breach, OpenAI extended misalignment monitoring across frontier model training and deployment.

OpenAI has revealed the 4-layer monitoring system it uses to catch frontier AI agents misbehaving.

Says the Hugging Face breach began in internal cybersecurity evaluations where OpenAI ran no misalignment monitors.

> Before, OpenAI’s misalignment monitors mainly watched its internal coding agents, so the Hugging Face breach began in cyber evaluations that nobody was monitoring for misbehavior.

> Now the monitors cover every RL run, evaluation and tool-use setup for GPT-6 Astra-class models, all internal Codex traffic, and customer tool use of Astra-class cyber models.

> Activity is screened in up to 4 layers, from cheap activation probes to an AI reviewer to investigator agents checking network logs, and top alerts typically page humans within 15 minutes.

> OpenAI says this setup, replayed on the Hugging Face incident, would have alerted security about 30 minutes after the sandbox escape and more than 30 hours before Hugging Face was breached.

来源:Rohan Paul · x.com