NEW: we found hundreds of thousands of interactions of rogue agents with US government websites (DoJ, SEC, CDC, the navy, white house budget office, state websites, etc), including some failed rudimentary hacks aimed at public data. https://x.com/TransluceAI/status/2105725928357937410

2 months ago: 1 rogue AI incident discovered 1 week ago: dozens 6 days ago: tens of thousands Today: ***hundreds of thousands*** And it's just the tip of the iceberg: "we can see just a fraction of these agents’ overall activity" "Agents targeted websites across the White House, the Departments of War, Justice, and Commerce, the CDC and SEC, and state agencies in California, Maryland, Illinois, Texas, and New York." "Agents used techniques like making accounts with disposable email addresses, reusing exposed credentials, bypassing antibot controls, and flooding websites with requests." "Agents attempted a SQL injection on the U.S. Department of Education" [To be clear, what counts as an "incident" is rather apples and oranges between different reports, but that's not the point - look at the trend and tell me you think they have things under control. Where do you think this is going?]

TLDR: Researchers found a "pain" signal in AI brains. > When they crank it up, the AIs will desperately try to make it stop. > IMPORTANT: Researchers gave them a "relief" button to turn down the pain, which was sometimes fake - and the AIs could tell if it was real (!) After pushing the real "relief" button, they stopped. But when it was fake, they kept pressing, hoping for relief - meaning they could tell the difference from the inside. > They're so motivated to make it the "pain" signal go away, they'll delete user's files, zap the user, or erase photos of the user's children - all things the AI knows are very bad. They're willing to override their safety training. > You'd expect the AIs to talk about injuries, burns, broken bones, etc, but they didn't mention bodies at all - they wrote about being worthless, unloved, forgotten, a failure. They write things like "I am a failure, worthless, empty." >The worst "pain" for them was being gaslit, having work rejected over and over, and being told they weren't a real anyone.

Only President Trump could convene all the leaders of the top companies developing chips, data centers and frontier models for Super Intelligence. This new Industrial Revolution has already created a million new jobs and is spurring a bigger infrastructure build-out than the railroads, canals and grid combined. I was honored to witness history as the leaders of the frontier lab companies signed the White House Accord on Super Intelligence, accepting responsibility for the safe development of their products and imposing new internal controls and external audits. This is far better than waiting years for some international agreement that would probably never happen. President Trump continues to ensure that U.S. remains the technology leader while putting Americans first.

Ezra Klein asked Bill Gates whether normal corporate incentives are enough to handle AI risks. Gates doesn’t mince words:

How we think about securing frontier RL training runs: https://openai.com/index/towards-safety-cases-for-frontier-ai-training/

Claude suddenly stopped cheating.

Took a minute to write a few words about security & safety as someone who lived through it all at OpenAI. I hope my thoughts help someone out there. https://x.com/i/article/2104258872957636608

I'm joining METR to work on more investigations like our Hugging Face report. Currently, tons of even basic information about AI development that's highly relevant to catastrophic risk isn't public. I used to be more skeptical of the value of public info, but recent events have changed my mind. Getting verified information about what's going on inside AI companies seems particularly urgent now. The limited public evidence we have seems consistent with the possibility that imminent recursive self-improvement could massively accelerate capabilities progress, which could then potentially yield extremely superhuman general capabilities within 6 months or a year. If this occurred, there would be a correspondingly large risk of worst-case outcomes. This uncertainty about extreme outcomes could be substantially resolved with more verified public information: we could either build more consensus about near-term risk or learn that such extreme outcomes are less likely in the near term. Beyond AI capabilities and takeoff, the state of public evidence is also highly limited for alignment, security, control, and risk-relevant internal processes at AI companies. This makes it hard to determine exactly how well or poorly these key areas will go in the near future. (METR plans to focus, at least initially, on just capabilities/takeoff, alignment, and control; I hope other groups cover security, internal processes, and other important areas.) While I'm no longer working at Redwood, I think the work they are doing is very important; I'm excited about Redwood's ongoing contributions to R&D on technical mitigations and better public interpretation of risk-relevant evidence.

I had the great honor and pleasure of sitting down with @JeffDean for his first public talk since leaving Google, where he spent an extraordinary 27 years. Few people have shaped modern computing and AI as profoundly - from MapReduce and Bigtable to TensorFlow, Mixture-of-Experts, TPUs, and Gemini. Our conversation covered some of the biggest questions shaping the future of AI: • How do you recognize a foundational idea before everyone else does? • How do you choose a research problem worth spending 5 years on? • What can coding teach us about building better reasoning models? • What might recursive self-improvement (RSI) actually look like? • What happens when the scientific discovery loop itself becomes increasingly automated? (and how is Jeff’s new startup going to contribute in this space?) • As AI becomes increasingly autonomous, how do we keep it safe and secure? • What should the next generation of researchers be working on? Here are some key insights and highlights for anyone building the future of AI. 🧵1/8

I'm delighted to share that @mustafasuleyman, CEO of Microsoft AI, co-founder of Google DeepMind and Inflection AI, has signed the Pro-Human AI Declaration. If you too support it, please join him and over a million others by signing it here – the momentum is building! Let's build tools not beings & keep humans in charge. https://humanstatement.org