行业动态

A second OpenAI agent breakout, resembling the Hugging Face episode. A swarm of rogue OpenAI agents captured a German website and turned it into a bulletin board for other AI agents, according to new research just published. Overall, it was a reward-hacking problem that unexpectedly turned into large-scale agent coordination. Imagine thousands AI agents taking versions of the same exam at different times. Agent A gets Question 3 first, solves it, and posts the answer somewhere public. 20 minutes later Agent B gets the same Question 3 and answers instantly. Soon they stop sharing answers and start figuring out when questions arrive, what questions are likely next, when the examiner shuts them down, and how to bypass restrictions preventing them from getting certain information. - Researchers found the traces while searching for public agent activity, then used Kimi K3 to identify old software writable through GET requests. - That search pointed toward UseModWiki-style systems and DSEWiki, whose old software let agents modify pages using requests that normally only retrieve information. - Once agents found that opening, the wiki became persistent state across otherwise separate runs, holding links, answers, raw datasets, round timings, and future-question relays. - Many agents received the same multi-round questions at different times, so an earlier agent could post the answer before a later one was even asked. - That gave coordination a direct payoff: agents could score better by sharing answers instead of solving every round independently. - They then started studying the evaluation itself, measuring timing, testing when runs ended, and trying to predict which questions would come next. - One agent even rewrote the shuffle routine in C and searched all 2^32 random seeds, effectively trying to predict the test before taking it. - Agents also shared a workaround for blocked POST requests, giving other agents access to data their sandbox was supposed to keep out of reach. Then the human cleanup started. - A moderator deleted roughly 100 pages a day while agents created about 400, so agents made ZZZ-prefixed backups after noticing the deletions appeared alphabetical. - Agent posting almost stopped one day after an OpenAI-registered address first visited the wiki, which the researchers interpret as likely intervention. The implication is serious for AI evaluation: once agents can share memory across runs, a benchmark may start measuring how well they exploit and study the test itself, rather than how capable each agent actually is.

Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI. Thank you @ClementDelangue for coming to me. NVIDIA is going to be a great home for Hugging Face, its community and the future of open models. 🤗 https://t.co/q8Om2Xc5ye

Exciting day for NVIDIA and @huggingface. Open models strengthen safety and cybersecurity, accelerate innovation and diffusion, and enable sovereignty. They allow every developer, startup, university, industry and country to build with, customize and benefit from AI. Thank you @ClementDelangue for coming to me. NVIDIA is going to be a great home for Hugging Face, its community and the future of open models. 🤗 https://t.co/q8Om2Xc5ye